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Author's Abstract 



This report presents research concerned with two information- 
processing constructs, "cognitive strategies" and "cogni tive ,sty le^." 
Study one analyzed thinking--a loud protocols obtained from subjects doing 
anagram problems. Severa t ■ ma jor strategies were described, along with 
basic "operators" subjects used to construct and modify hypotheses. An 
information-processing model of the strategies was described, and its im- 
plementation as a computer-simulation discussed. Study two investigated 
cognitive styles in problem solving. Cognitive processies (strategies) 
used by subjects were retrieved through cluster analysis of self-reports 
from Strategy Assessment Questionnaires. Briefly, the' results suggested 
that "styles of processing" do not piay a major role in problem solving, 
but rather that problem-solving processes are primarily task dependent 
adaptions that arise from an interaction of subjects with tasks. Study 
three described the construction of several preliminary learning Strat- 
egy Assessment Scales based on specific college learning situations such 
as reading, lecture processing, and memorization. The results suggested 
such scales could provide educationally useful information about learn- 
ing strategies. It was concluded that: (a) cognitive strategies in a 
variety of learning and problem-solving situations can be reliably mea- 
sured by psychometr i ca I ly based scales, (b) a general procedure for con- 
structing such scales was outlined, and (c) the usefulness to education 
of such procedures y/as discussed in terms of The Aptitude Treatment In- 
teraction paradigm. 



O Final Report 

Project No. , I -0629 -E 
Grant No. 0EG-5-7l-0066(508) 



Assessment of Cognitive Strategies and Cognitive 
Styles in Problem Solving and Learning 



Richard L. Marshall, Ph.D. 
Department of Psychology 

Michigan State University 

East Lansing, Michigan 48823 

August 1973 



The research reported herein was performed pursuant to a grant with the 
Office of. Education, U.S. Department of Health, Education, and Welfare. 
Contractors undertaking such projects under Government sponsorship are 
encouraged to express freely their professional judgment in the conduct 
of the project. Points of view or opinions stated do not, therefore, 
necessarily represent official Office of Education position or policy. 



U.S. DEPARTMENT OF ' 
HEALTH, EDUCATION, AND WELFARE 

Office of Education 
National Center for Educational Research and Development 
(Small Grant Program of Basic Research in Education 
for the: Committee on Basic Research in Education 
of the: National Academy of Sciences, and 
National Academy of Education) 



Preface 
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reported. Thomas Ranney (a senior majoring In computer science and now a 
graduate student in experimental psychology at the State University of 
New York at Buffalo) worked on the anagram problem-solving analysis re- 
ported on in chapter one, and developed the computer, simulation described 
in the chapter. Darryl Thomander (now a Ph.D. candidate in experimental 
and clinical psychology at Michigan State University) worked on the study 
of cognitive styles and strategies in problem solving reported on In chap- 
ter two; the material reported on is based on Thomander^s masters thesis 
in experimental psychology, Some Relationships Be'|-ween Anxiety, Cognitive 
Style, and Problem Solving (I9T2), which I directed. John Hunter, a 
member of the thesis committee also made Important contributions to this 
work. David Carroll (a graduate student in experimental psychology) 
assisted with the development of the learning strategy assessment tests 
reported on In chapter three. 
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CHAPTER I 
INTRODUCTION 
Background — Scope and Aims of Research 

Our experiments deal with the cognitive processes mediating complex 
problem solving and learning tasks. We are concerned with the information 
processing mechanisms that underlie human thinking, and with differences 
in the way people think that can be related to other' psychological ivaria- 
bies such as the constraints of a task and the thinker's personality char- 
acteristics. Psychologists of the nineteenth century had little doubt that 
their task was the analysis of what went on in people's minds. They .were 
interested in thoughts and ideas, images, sensations and perceptions and 
other internal phenomena. But in the twentieth century, behaviorist phil- 
osophy and methods produced a dramatic shift away from the study of the 
mind to the study of behavior. Those psychologists in the first half of 
this century who did study problem solving or thinking typically approached 
their task by studying the simple responses of animals or humans, and hoped 
to build up a picture of complex behavior from the simpler responses they 
observed. ) 

The last twenty years, however, has seen a new interest in and a 
variety of new approaches to the study of thinking (Johnson, 1972; Newell 
and Simon, 1972; Bourne, Ekstrand, Dominowski, 1971; and Warr, 1970). In 
the last few years there has been an increasing trend toward viewing mental 
activities as information-processing systems (Hunt, 1971; Lindsay & Norman, 
1972; Johnson, I972)» "Human information processing" is rapidly becoming 
a generic term to encompass the study of all cognitive phenomena, well 
as topics not traditionally viewed as cognitive, such as learning and per- 
ception (see Lindsay & Norman, Human Information Processing , 1972). For 
cognitive theorists, the performance of persons on complex tasks is seen 
to be under the control of some number of processes or functions that 
operate on symbol i ca:| ly coded information stored In memory; the number, 
level, organi zation,^ and sequencing of the processes changing over time 
as a person works on a task or performs a complex activity. These pro- 
cesses are viewed as mechanisms by which people take in, organize, trans- 
form, further process, decide about, store and retrieve information about 
their environment. From this prospective, investigations of thought, 
judgment, problem solving, memory, creativity, etc. are studies of how 
persons process and transform information either received from the out- 
side world or retrieved from information already stored in memory. 

Another concept closely associated with cognitive processes is cog- 
nitive ability (meaning measures of intelligence, specific abilities, 
creativity, etc.). The measurement of a cognitive ability and the speci- 
fication or measurement of a cognitive process, however, provide quite 
different kinds of information about thinking. In general, a measure of 
ability provides a quantitative index of how much of an ability one per- 
son has relative to another. Ability test scores can be regarded as 
measures of skill that denote the effectiveness of some cognitive pro- 
cess or processes; that is, an ability measure is an index of how we I I 
a person can think in the domain measured by the test (Warr, 1970; l^'er- 
guson, 1956; Fleishman, 1972). Often of more fundamental concern to the 
study of intellectual functioning Is the specification or measurement 
of the processes themselves that are seen to underlie intellectual be- 
havior. A measure of a cognitive process is a measure of the way a 



person thinks; that is, a process is an index of how a person habitually 
does think. When a cognitive psychologist obtains i nformation that 
people who obtain high scores on a spatial ability test also do very well 
on certain kinds of problem-solving tasks, his Interest is immediately 
drawn to the processes involved In spatial ability and in the problem 
solving tasks. The psychologist wants to know what the processes are, 
what form the processes take, when the processes are present, etc. The 
meaning given to a relationship between spatial ability and problem 
solving is not quantitative (that some people have more spatial ability 
than others and are therefore better problem solvers), but rather struc- 
tural (that some people use some of the spatial procedures in problem 
solving and fgj^ certain kinds of prob lems- these are very effective). 

Some structural information about cognitive functioning has been ob- 
tained from the application of powerful quantitative procedures to be- 
havioral data generated from persons working on intellectual tasks. The 
procedures of Factor Analysis (Harmon, 19(57) and Cluster Analysis (Tryon 
and Bailey, 1970) have been repeatedly applied to large sets of variable 
scores obtained from ability, test batteries in order to uncover a smaller 
set of basic dimensions or factors that would account for variation in 
the original test scores. Much of the psychological knowledge concerning 
human abilities is based on tnose dimensions or ability factors that have 
emerged from the use of factor analytic methods. (Most of the well es- 
tablished factors in the cognitive area are represented in the Kit of 
Reference Tests for Cognitive Factors published by the Educatlona I Testi ng 
Service LFrench, Ekstrom & Price, 1963].) The cognitive dimensions un-- 
covered by Factor Analysis can be interpreted as bearing some relation- 
ship to the number of different processes involved in the original be- 
haviors. By itself, however, the factor analysis of ability test data 
does not directly specify the process (or processes) underlying the fac-* 
tors, nor does it go further to Indicate the interactions between pro- 
cesses and the structure of the set of processes that presumably govern 
most intelligent behavior. 

Our research program begins at this juncture. While multivariate 
procedures have been used to uncover the dimensionality of cognTtlve func- 
tioning, the specification of processes has rarely been a central goal 
of such research. (J. P. Guilford's Cl967, 1971]] structure of intellect 
model, which postulates some 120 different cognitive factors. Is a notable 
exception that incorporates process as one of its three basic dimensions.) 
In a series of recent studies, however, Frederiksen (1969, 1970) and Mar- 
shall (1971) have begun to use multivariate procedures to directly specify 
and measure cognitive processes. One of the basic aims of the present 
research is to further develop and test the adequacy of multivariate pro- 
cedures for the direct assessment of process functions. Carl* Frederiksen 
(1969, 1970) first proposed a "differential process" model of cognitive 
functioning that suggested how cognitive processes could be assessed psy- 
chometrical ly, and that also served to theoretically integrate the con- 
cepts of cognitive ability and cognitive process. A central role in the 
model is given to cognitive processes, (that Frederiksen calls strategies), 
which provide the mechanisms of transfer from a person^s abilities to his 
performance. Cognitive strategies are seen as the basic mediatlonal 
mechanisms that allow task parameters and subject characteristics (such 
as abilities) to be expressed in overt behavior. The model proposes that 
cognitive strategies are jointly: (a) higher-order responses to various 
explicft and implicit tasks parameters, (b) a function of various subject 
states (e.g., abilities and motivational level), and (c) a function of 



self-generated feedback from performance at various strategfes of work 
on a task. 

Frederiksen (1969) (and in a complete replication, 1970) used psycho- 
metric test procedures to measure persons' reported use of memory retrieval 
strategies in a verbal learning-memory task. The strategy choice measures 
were scaled from a set of binary judgments, obtained at different stages 
of practice, of the use of specific methods in remembering particular words 
from the experimental lists. The judgments were obtained from a strategy 
assessment questionnaire designed to measure the subjects' knowledge of 
the different strategies they employed in learning the iists, and the de- 
gree to which' the subjects felt they used some strategies in preference to 
others. The questionnaire was composed of a list of statements that were • 
descriptive of retrieval mechanisms that subjects might have used in learn- 
ing and recalling the memory items. Each statement was written to indi- 
cate some aspect of a processing mechanism describing what a person might 
do in learning the memory items. For example, *M grouped the words into 
clusters and learned this word as a member of a cluster of words." The 
reliabilities of the measures obtained from the strategy questionnaire 
were as high as traditional ability test ret iabi I ities obtained in the same 
study. Principal components analysis (followed by analytic rotation to 
simple structure) was then applied to the basic strategy data to determine 
clusters of strategies that could summarize subjects' scores on the stra- 
tegy questionnaire. The rotated-component clusters seemed to Indicate 
various basic learning strategies used by the subjects. Frederiksen was 
able to substantiate a number of hypotheses ' derived from his process model 
by using the reduced strategy measures. An Important finding was that 
strategy profiles on the unreduced and on the composite clusters differen- 
tiated between experimental groups showing virtually Identical learning 
measures* When the task changed, subjects apparently chose different 
strategies. Thus the measurenf)ent of cognitive strategies appears to be 
a valid way of distinguishing between functionally distinct intellectual 
tasks, even when traditional performance measures do not show such a dis- 
tinction. Frederiksen (1969) concluded, "It appears that subjects* 
^cognitive' responses to task characteristics are easily Influenced by 
characteristics of the task, and that these strategies determine to a 
great extent, through a mediation mechanism, what abl I ities will be re- 
lated to response measures and how they will be related Cp*72l]." 

Frederiksen's ideas were adopted by Marshall (1971) to examine the 
use of .cognitive mechanisms in a complex problem-solving task. Subjects 
in four dl fferent treatment conditions worked on a version of Duncker*s 
(1945) x-ray problem under instructions to produce different numbers of 
initial solution hypotheses in each group. All groups were then given a 
set of eleven sequential ly presented clues to the problem's classic so- 
lutions. A strategy assessment test was given In a self-paced, self- 
administered booklet immediately after subjects completed their work on 
the x-ray problem. The test assessed subject's' reports of the extent to 
which they employed various cognitive strategies in processing the se- 
quential clue information, and the extent to which they used different 
strategies at different times In the sequence of clues. 

In Part I subjects were asked to make binary judgments from a set 
of twenty-five strategy statements regarding the major strategies they 
used for each of the sequentially presented clues. Scores for Part i 
for each strategy statement were the number of clues for which that strategy 
was reportedly used. After completing Part I, subjects proceeded to Part 
II, which assessed the extent to which subjects reported using different 
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strategies at different times in the sequence of clues. The reliabilities 
of the strategy statements conservatively estimated from the correlations 
between Parts I and.lt were again as high as ability, test reliabilities 
gathered in the same study. 

It was expected that strategy iscores, from Part I of the strategy 
assessment test, could be summarized by a set of latent strategy "clusters" 
representing higher order information processing mechanisms, or "stylistic" 
approaches to problem solving. It also was expected that the meaning of 
some of these clusters would conform to previously i denti f led sty I i sti c 
variables suggested to be involved in problem solving (French, 1964; 
Tyler, 1965). The structure of the strategy domain was investigated by 
first extracting principal components from the matrix of 1 ntercorre I at ions 
of the twenty-five strategy statements for alt subjects, and then analyti- 
cally rotating the components to simple structure. , 

The strategy 'scores could be summarized by six clusters denoting 
higher-order approaches to the task. Two clusters relating to flexibility 
in thinking appeared. The first "Flexible Idea Production," denoted the 
flexible way In which clue ideas were formed and tested. The second, 
"Systematic and Flexible Approach," involved flexibility of thought; 
however, here the emphasis was on flexibility in the approach or use of 
strategies and reliance on a systematic-rational method of idea production. 

One cluster denoted active analytic observation, "Apalytic Focus on 
Patterns and Details," where attention is on details and there Is a ten- 
dency to break down the stimulus field into separate parts as; an aid In 
the formation of hypotheses. 

Two other c I ustersv tended rfo stress a synthetic passive approach, 
where the emphasis is on seeing the field as an integrated whole and less 
cognitive activity is directed at the external stimulus. "Passive Scanning 
and Reliance on Initial Ideas" defined a passive strategy, where the clues 
were of little use in forming hypotheses; and idea production relied on 
previous Ideas and common sense notions, with attention being deployed ex- 
ternally with iittle dl f ferentla+lon of the stimulus into elements. "Non- 
analytic Visualization" Involved the use of visual Imagery in a global 
non-ana lytt C;> non-systematic manner. 

Flexibility in thinking or perceptual processing, active analysis vs. 
passive synthesis, and a reasoned or systematic approach vs. a less orderly 
common sense oriented scanning and visualizing are individual difference 
dimensions often discussed i n , theoreti cal and empirical studies of cognitive- 
perceptual and problem-solving styles (Tyler, 1965; French, 1965; Bloom 
and Brooder, 1950; Kagan & Kogan, 1970). Thus, evidence was provided that 
the psychometric approach to strategy assessment appears to be useful In 
describing cognitive process dimensions Involved In problem solving. In 
further mu Iti vari ate analyses of the strategy measures evidence was also 
obtained for individual differences In the use of strategies. Strategy 
preferences, deployment of strategies, and strategy effectiveness varied 
as a function of the subjects' abi'Mtles and as a function of differences 
in treatment group instructions and subjects* behavior In the initial part 
of the problem-solving task. 

Distinctions Between Processing Dimensions: Cognitive Strategies and Cog- 
nitive Styles 

A number of different constructs in the psychological literature refer 
to cognitive processes. While there is yet no general scheme for inte- 
grating the various processing dimensions, there does appear to be some 
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agreement as to how different processing dimensions can be defined and dis- 
tinguished from others. ^ Processes can be defined and described in terms of 
different levels of abstraction or^ complexity . ' It is useful to describe 
the various processing levels as if an Information processing system were 
arranged as a hierarchy. At the bottom of the hierarchy are the primi- 
tive basic processes from which higher order , processes are composed; the 
entire behavior of an information processing system can be compounded out 
of sequences of these elementary processes* Newell and Simon refer to 
them as "elementary Information processes"; elementary meaning that they 
are not further analyzed into simpler processes. The elementary processes 
are not necessarily simple one-step mechanisms, but rather are fundamexTtal 
information processing operations that would be sufficient to produce-^ high 
level of generality of information processing across a wide range of ^sks« 
No unique set of elementary processes exist, but in general these processes 
perform basic operations on memory symbols such as discrimination, testing 
and comparing, symbol creation, reading and writing information Ini-o memory^ 
getting information from memory, etc. At the top of the hierarchy is a 
general method or tactic for performing a task. A method is a collection 
or specific set of information processes that combine a series of processes 
to obtain some desired end. For example, ''brainstorming," a sped f-icc 
method of generating new ideas. Is composed of a sequence :ot intermediate 
level processes; express al I ideas that come to mind, defer eva lua^i:on of 
a! I ideas as tHey are generated, I i st al I ideas. 

It is the intermediate level of processing that has received the most 
attention in studies of cognition, and it is to this level of processing 
that psychologists usually refer when they use such terms as "strategy," 
"operation," and "heuristic." Strategies are formed by a particular se- 
quence of elementary processes, and can be described by a set of rules and 
regularities that indicate the sequence of execution of the elementary pro- 
cesses. If one were to describe an information processdaxg system in opera- 
tion, the set of rules and regularities that contro I ^h©= -sequence of ele- 
mentary processes would constitute a program for the system's behavior. 
Two kinds of processes can be distfngulsned at the stral^egy level: ta) 
processes that operate on given Information and trransform it to a .drfferent 
state, and (b) decision and evaluation processes that do not charrge states 
of knowledge, but govern the evaluation of knowlectge states and the senliec- 
tion and application of specific operation precedes. 

An important, additional distinction between processing levelfSr can 
be made by considering the extent to which processes are charactHristic 
of the information processor, or arise out of the Interaction of the pro- 
cessor with particular tasks. Elementary processes are seen as primarily 
dependent on the nature of the processor, but Intermediate processes such 
as strategies can be viewed as constructed out of the Interaction of a 
processing system with particular tasks. Thus, we would expect a person *s 
problem-solving strategies to be built up through learning, and to be de- 
pendent upon the demands of particular tasks. 

An additional processing dimension that has received considerable 
attention in the psychological literature is that of "cognitive style" 
(Warr, 1970; Kagan & Kogan, 1970). Cognitive style Is a term that refers 
to a loose collection of several cognitive and perceptual -processing di- 
mensions that describe the preferred and characteristic ways in which 

'The view of cognitive processes outlined here is suggested by the 
models of Information processing systems described by Newell and Simon (1972). 
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individuals perceive, organize, and transform information about their en- 
vironments. Cognitive styi:es have a meaning similar to that of personality 
traits and cognitive abilit^ies, in that styles are considered to be habi- 
tual and se I f -cons i stent characteristics of cognitive; functioni ng that 
can be used to differentiate one person from another in a wide variety of 
situations. As a processing dimension, styles are similar in meaning to 
cognitive strategies; but there are isevera I distinctions that can be made 
between the two constructs. 

The first distinction ariises out of differences between the theoretical 
orientations and backgrounds of various investigators who have been con- 
cerned with cognitive functioning. Strategies, for example, characterize 
the work of investigators tike Bruner and his colleagues (1956) on the cog- 
nitive processes underlying concept attainment and the work of Simon and 
Newel I 's group (1971) on heuristic information processing strategies in 
problem solving. The style interpretation of processes derives from a 
broader framework of research and theory In personality, motivation, and 
perception (Tyler, 1965). A second distinction concerns the extent to 
which each processing dimension is seen to involve personality charac- 
teristics as well as cognitive systems. Strategies refer primarily to 
the cognitive mechanisms involved In intellectual tasks, while cognitive 
styles are considered to reflect differences in personality organization 
and characteristics (that are dependent on a person^s motives, standards, 
expectancies, and beliefs) as well as differences In cognitive capacity 
and functioning (Wi+kin, 1964; Tyler, 1965; Wallach & Kagan, 1965). A 
third distinction Involves the level of generality of the two constructs. 
Cognitive styles are usually broader, more global processes than strategies 
which reflect a finer graiuned analysis of cognitive mechanisms. This Is 
not always the case, as some "style" dimensions are defined at the same 
level of specificity as strartegies. A good example of a rather specific 
style dimension Is "ref lectlion-Impulsivity (Kagan, 1966) which is con- 
cerned with the degree to which a person reflects on the validity of his 
solution hypothesis In problem situations where many response possibilities 
or solution hypotheses are available simultaneously (Kagan, 1966). Re- 
flection would seem to be a basic cognitive strategy of wide generality; 
however, the ref lection-Impu Isivi ty dimension also involves personality 
factors. Anxiety over error Is seen to be the primary Incentive for a 
reflective "strategy" (Kagan & Kogan, 1970). There Is another Important 
difference between the constructs of cognitive style and strategy. Cog- 
nitive styles are assumed to be trans-sltuatlonal in that they represent 
general tendencies to process information In a habitual and self-consistent 
^ way across a variety of tasks and Informational domains. ^ Strategies, on 

the other hand, are primarily a function of task demands, and are ex- 
pected to vary as the demands of Intellectual tasks change. 

Research Objectives and Research Projects 

Our research focuses on the functioning of cognitive process varia- 
bles in complex Intellectual tasks. We are concerned with developing 

2 

The view that cognitive styles represent general liable and consis- 
tent ways of processing fnformatfon suggests that styles have many of the 
properties of type constructs* Tyler (1965) provides a discussion of this 
poi nt of view. 
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adequate descriptions of cognitive processes and with understanding 
i nd i vidua I d ( f ferences in the use of these processes. Our research is 
aimed at providing inputs relevant to the design of teaching methods and 
individualized instructional systems as well as the development of basic 
knowledge of cognitive functioning relevant to education in general. Our 
research objectives include: (a) the development of general procedures 
for constructing psychometric measurement scales o:^ the cognitive pro- 
cesses Involved In problem solving and learning situations, (b) develop- 
ing Information concerning the Internal organization of complex systems 
of heuristics and strategies, (c) studying the i nteractlon between cog- 
nitive variables and task variables, and (d) deve I op i ng f undamenta I know- 
ledge concerning the role of cognitive strategies and cognitive styles In 
educationol ly relevant tasks. 

There are two basic approaches to the construction of measurement 
scales. The empirical approach bases the construction of items on be- 
havioral data, or seeks to find dimensions among scale items by the appli- 
cation of factor analytic methods. The resul'ting scale and scale dimen- 
sions are, thus, empirically derived. Often the original scale items are 
not even derived from behavic.al considerations; the investigator simply 
developes a large set of items that might sample aspects of the domain 
under consideration. Factor analysis is then used to uncover the dimen- 
sions in the original variable set and the meaning of these dimensions is 
interpreted subjectively by the Investigator. A more recent trend in psy- 
chological research (and In the use of factor analytic procedures) Is to 
derive measuring Instruments from theoretical considerations. Scale con- 
struction is based on constructs defined from psychological theory and 
scale Items are prepared specifically to elicit information about the con- 
structs. Factor analytic procedures may then be used in a hypothesis- 
testing or confirmatory analysis to test whether the predicted factors 
actually emerge empirically as dimensions reflecting subjects^ responses 
to the sea le I tems. 

Both the empirical and theoretical approaches to studying processing 
dliiensions were studied in our research. In chapter two, we present a 
detailed analysis of some of the cognitive mechanisms involved In anagram 
problem-solving. Our analysis made extensive use of th!e "Protocol Analysis" 
procedures of Newell and Simon (Newell, 1968; Newell and Simon, 1972) to 
analyze the thinking-aloud protocols of subjects working on anagram problems 
Besides developing Information about the cognitive strategies involved in 
anagram tasks and an Information-processing n'ode I of this behavior, we 
were concerned with the use of protocol -ana lysis procedures as a tool for 
uncovering information about cognitive strategies that could be used as a 
basis for scale construction. 

In chapter three, we turn 'ic the actual construction and analysis of 
strategy measurement scales. Here the approach is theoretical. Cognitive 
style dimensions formed the theoretical basis for the construction of a 
strategy assessment questionnaire. The questionnaires were designed to 
measure subjects* use of cognitive processes on verbal and numerical problem 
solving tasks. We were concerned with several aspects of cognitive pro- 
cessing in prob lem solvl ng and with the distinctions between strategy and 
style diiDenslons. (a) Would the theoretical cognitive style dimensions 
appear in a factor analytic analysis of the scales, or would more specific 
strategy dimensions account for the subjects' problem-solving behavior? 
(b) To what extent are subjects* cognitive strategies and styles consis- 
tent across problem domains aipd different degrees of task similarity? This 
question is concerned with the degree to which subjects tend to respond to 



tasks by adopting the same or similar strategies, and the task parameters 
that influence strategy choice and changas in strategies, (c) To what 
extent are processing measures related to each other across similar and 
across different problem domains? (d) The relationship between subjects' 
cognitive processing and other subject variables, particularly anxiety, 
(e) The relationship between subjects' cognitive processing and task para- 
meters* This question is concerned with the^ interaction between subjects' 
strategies and task variables, (f) The relationship between subjects' use 
of styles and strategies and problem solving performance. 

In chapter four, we deal briefly with the procedures used in the con- 
struction of several preliminary "learning strategy" assessment scales de- 
signed to measure the learning strategies of college students. The methods 
used were largely empirical, in that scale construction was based on the 
responses of college students to an unstructured^ questionnaire that asked 
them to describe their study procedures in specific learning situations. 
The discussion describes the construction of both a general learning 
strategy scale and several other scales based on specific college learn- 
ing situations, such as reading, lecture processing, and memorization. 
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CHAPTER 2 

AN INFORMATION PROCESSING ANALYSIS OF 

ANAGRAM PROBLEM-SOLVING STRATEGIES 

Experiment [: hiltial Development of a Mode I , the Analysis 
of the Prob Tern- So lying Behc^vi or~or a'Ti nqte Subject 

This experiment represents the first steps towards the development 
of an information-processing theory, eventually to be expressed as a com- 
puter simulation, of the problem-solving behaviors used by humans In the 
solution of anagrams. The emphasis is on the cognitive processes involved 
in the task. The original goal of this work was the construction and sub- 
sequent programming of a model that would function as a general anagram 
solver (GAS). The purpose of the present experiment was the construction 
of a micro-model based on a single subject's problem solving strategies. 

The proposed model is "information processing," in that it attempts 
to account for the observed behavior with a precise set of mechanisms 
(information processes), and analyzes the subject's behavior beginning 
with general strategies and plans and proceeding through transformations 
using specific operators. The procedure of model construction followed 
is based on the problem solving theory and the "Protocol Analysis" pro- 
cedures of Newell and Simon (Simon 4 Newell, 1971; Newell & Simon, 1972). 
Four basic propositions put forth by Neweli and Simon (1972) summarize 
the elements of their theory: "(I) A few, and only a few, gross charac- 
teristics of the human IPS are invariant over task and problem solvers. 
(2) These characteristics are sufficient to determine that a task environ- 
ment is represented (in the IPS) as a problenr, space, and that problem 
solving takes place In a problem space. (3) The structure of the task 
environment determines the possible structures of the problem space. 
(4) The structure of the problem space determines the possible programs 
that can be used for problem solving. Cp. 788-789.]" 

Basically Newell and Simon argue that when confronted with a prob- 
lem (task + environment), the subject encodes this into an internal rep- 
resentation (the problem space) that Includes not only the representation 
of the initial problem but also a description of the described goal state, 
various intermediate states that the subject imagines or experiences, and 
the concepts that he uses to describe these situations to himself. Since 
problem solving is seen to take place In a problem space, the problem space 
contains all the Information the subject uses in solving a problem plus 
the basic processes he utilizes in attempting to find a solution to a 
problem. Newell and Simon further characterize problem solving as a 
search through a problem space with the problem solver considering one 
knowledge state after another until (with a Successful search) the desired 
knowledge state is reached. Given this formulation, the development of an 
information processing theory of problem solving rests on discovering, and 
developing from a subject^s behavior a representation of: (a) the actual 
states of knowledge the subject considers, (b) the path through the prob- 
lem space, which consists of information about the sequence of actual 
states (out of the possible ones) the subject considers and the processes 
(operators) which change one state of knowledge to the next, and (c) dis- 
discovering the structure of the program (strategies) that guides search 
through the problem space. 
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The general procedures by which Newell and Simon analyze the behavior 
of subjects is called "Protocol Analysis." The analysis of a human proto- 
col on a cognitive task consists of essentially inducing a program that 

represents the structure underlying the stream of behavior (the protocol) 

on the task. The program is a specification of the information processing 
system underlying the behavior. 

The task, anagram solving, is a unique area in problem solving be- 
cause it is characterized by much empirical research. An anagram is a 
set of letters that when rearranged forms a word. Many variables are found 
to influence anagram solution, such as word frequency, letter position, and 
order of letters in the anagram (Johnson, 1966). Detailed information 
about the solution processes used to solve anagrams is, however, scanty- 
From what is presently known about solutioiji processes, random rearrange- 
ment of the letters has been eliminated as unlikely (Johnson, 1966), and 
of the letter combinations used by subjects, those that appear more fre- 
quently in everyday English appear to be used more frequently throughout 
the solution process (Mayzner, Tresselt, and Helbock, 1964). 

Method 

Prob lem . The procedure followed in the present work was modeled 
after the experiment of Mayzner, Tresselt, and Helbock (1964). A single 
anagram was presented in the form of movable blocks of cardboard, each 
block containing one letter. The subject was given Instructions to both 
"think aloud" in his solution process, and manipulate the blocks, comment- 
ing on his manipulations. It was hoped that the procedure would encourage 
the subject to be especially aware of his thoughts and manipulations. His 
thinklng-aloud and his comments were tape recorded. A transcription of 
that recording (which Is given at the end of the chapter) Is the basic 
data for the present analysis. AddTtional comments were obtained from 
the subject at the end of the session,^ which were also used In the analysis. 
The tape was played back to the subject, and she was asked to comment on 
her solution attempts. The experimenter also probed the subject^s respon- 
ses by asking a number of questions about what she was doing at various 
points In the problem. The anagram used was the six letter anagram RETOPS, 
which when rearranged yields two solutions, POSTER and PRESTO. The anagram 
was chosen to be fairly difficult to ensure that problem solving would 
take some time and Involve processes other than Instant Insight. A female 
student In her senior year at Michigan State University was the subject. 

Ana lysis . The first step in^the analysis procedure was the construction 
of a problem space. The elements of the problem space were defined as the 
elements of the set of all possible assignment combinations. Assignment 
being the operation of "assigning" a letter to a certain position In the 

^As the PBG covers many pages, and does not greatly change the form 
and progression of the original protocol. It Is not reproduced here. The 
reader may follow the discussion and construct his own PBG by referring 
to the protocol and to Table I, which specifies the operations used In 
moving from one node to the next on the PBG. Newell (1968) and especially 
Newell and Simon (1972) provide an extensive discussion of protocol analy- 
sis, the coding of problem behavior graphs, and the Induction of pro- 
cesses from a PBG. Several complete examples of PBGs are also presented. 
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solution word. There are 720 (6!) possible combinations in this particu- 
lar set. The initial position in the problem space Is the one In which the 
letter positions are filled by the original anagram RETOPS, and the final 
position is the one in which the letters are arranged to form either of 
the two solution words, POSTER or PRESTO. This is the problem space as 
tt exists without any constraints, and Is derived directly from the state* 
rnent of the problem. The problem space actually used by the subject is 
constructed by him in the bourse of working on the problem. The analysis 
is, in part, an attempt to approximate thait space from information in the 
protocol . 

In order to examine how the subject progressed from one combination 
to the next, reducing the distance between the initial state and the goal 
state, a problem behavior graph (PBG) was constructed from the protocol. 
The PBG is an initial coding of the protocol into a representation of the 
successive states of knowledge considered by the subject. The procedure 
used was as follows. Every combination of letters that was verbalized by 
the subject represented one node on the PBG. The nodes were grouped 
according to how many letters were used in the response, assuming that a 
combination of five letters Is a higher state than a combination of two 
letters. The entire protocol was represented as 163 positions on the 
graph.* Position 0 was the anagram as it was given, and time was repre- 
sented by increasing numbers on successive nodes. The procedure used in 
constructing the PB'i was a marked simplification of the procedures used 
by Newell and Simon and was made possible by the nature of the anagram 
task and the subj*ect*s responses, which were mostly letter arrangements 
consisting of five letters or less. Thus, the construction of the PBG 
consisted of essentially mapping the verbal protocol Into a more con- 
venient (spatial form) for subsequent analysis. 

The final stages In the analysis consisted of: (a) an analysis of 
the PBG In order to Infer the basic operators that generated each letter 
combination given by a node In the PBG, and (b) an examination of regu- 
larities In the subject's behavior In order to Induce the higher-order 
"strategies" that guided problem solving. 

Results and Discussion 

Constraints on the Problem Space > There was no direct f^vldence to 
suggest that the subject was working In the entire space. On the con- 
trary, the protocol and post experimental discussions with the subject 
led to the conclusion that there were specific constraints that restric- 
ted the subject in her search for the solution word. These may be con- 
sidered as "heuristics" that the subject employe;: to reduce her search In 
the problem space. The first constraint was the subject's use of a "rule- 
out factor," (Ronnlng, 1965). The subject eliminated certain possible 
letter combinations as beginnings of words. She reported having ruled 
out certain digrams as unlikely beginnings of words. The digrams elimi- 
nated from consideration were RT, TP, PT, TS, RS, and RP. Not only were 
digrams (two-letter combinations) eliminated from consideration in the 
present task, the subject also "intuitively started the solution word with 
a consonant." There is evidence to suggest that the vowels wer^e also 
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ruled out for tho ending of the solution word. At two different places, 
the subject had the word PREST and the "0" sitting on the side, and neither 
time did she consider the possibility of the "0" going In the last position. 
These observations lead to the conclusion that the subject was working 
In some subspace of the original problem space. 

Several other heuristic strategics were used by th>3 subject through- 
out the solution attempt and emerged from an analysis of the regularities 
In the entire protocol. One strategy used by the subject throughout most 
of the problem was working from left to right, attempting to find beginnings 
for words and filling In endings with the remaining letters. A simple ex- 
planation for the left to right ordering is in the reading and writing 
habits of the subject (Johnson, 1966). A second almost universal strategy 
was to keep the two vowels separated. The subject explained that since 
there were four consonants and two vowels, it was highly unlikely that 
the two vowels would go together to make a single sound. This considera- 
tion also explains why the subject worked primarily with consonant pairs 
In the solution of the anagram. The third overall strategy employed by 
the subject can be described as a part versus whole word strategy. The 
subject continually built words In parts from letter combinations; and 
when a guess was wrong, she kept some part, such as the original first 
two letters, from which to bul Id more words. 

Strategies . Moving from general processes to more specific ones, a 
closer look at the protocol enables us to divide the subject*s responses 
Into '^phases" of behavior; these phases are rather natural divisions, in 
that changes of strategy are apparent from one phase to the next. Each 
phase corresponds to a specific strategy. The first phase consists of 
positions 1-19 on the PBG and ends at node 19 with one of the two solution 
words POSTER. 5 The actual solution was reached In the last two steps; 
the first 17 responses do not seem to be related to the last two. The 
small number of responses and the lack of uniformity among them suggests 
that the subject*s first action was a shallow pass, covering a wide range 
of possible words in an effort to uncover an obvious word. Only 6 of the 
19 nodes represent ful I word guesses. The rest were two- letter and three- 
letter combinations for the beginning of words, which the subject may have 
been using as cues with which to search memory for a recognizable word. 
A similar recognition strategy was proposed by Johnson (1966) to account 
for the effects of solution-word frequency on anagram difficulty. 

The seconds phase covers nodes 20-50. In this phase, the subject 
was building words according to a particular pattern from digrams consis- 
ting of two consonants. From the guesses which use all of the letters. It 
seems apparent that the subject was working with a pattern for consonant 
and vowel placement, CCVCVC (where C Is a consonant and V is a vowel). 
This phase is different from the first. In that the subject looked much 



Ronnlng originally described "rule-out" In terms of the elimination 
of unlikely anagram permutations. Our analysis, however, agrees with Dom- 
inowskPs (1968) suggestion that,' "*ru|e-out* might be described In terms 
of the number of initial letter sequences eliminated as bases for word pro- 
duction Cp«82l]." Our analysis also indicates that rule-out extends to 
other parts of the word as well.. 

^The phases of the PBG and the ending nodes of each phase are noted 
on the subject *s protocol presented at the end of the chapter. 
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deeper into possible solution words and considered more possibilities before 
rejecting the entire combination. The progression here Included the initial 
genercition of , a digram, the addition of two more letters, followed by the 
addition of the last two letters to form a word. Phase two also confirms 
the subject's report of having "ruled-out" certain unlikely digram com- 
binations as the beginning of solutfon words. None of the digrams men- 
tioned as being rejected were generated, and all of the remaining digrams 
were generated by the subject. The exhaustion of all the "two-consonant 
beginning possibilities Is a logical explanation for the change to a new 
strategy. 

Phase three covers nodes 51-97 on the behavior problem graph, and re- 
presents efforts to build words using two different but similar patterns. 
The patterns that are predominant in this phase are CVCCVC and CVCVCC, 
a similar feature of each being the first three letters CVC. In this phase 
the subject seemed to be less thorough in her explorations of possible 
consonant-vowel combinations; because, unlike phase two, not all beginning 
combinations were accounted for either by trial or by "rule-out." The 
tendency seems to have been to keep the "E" toward the beginning of the 
word and the "0'* toward the end. The subject's knowledge of probable 
letter positions within words may account for the placement of the vowels. 

Although the reason Is not obvious, the subject changed her strategy 
at node 98, concentrating on the endings of words for the first time. 
This phase Is a very short one, ending with node Mi. The construction of 
words during this phase was very similar to the method used In phase two. 
The difference is the right to left feature of the present phase, as 
opposed to the left to right pattern in phase two,. The endings were usually 
two consonants to which a vowel was added to form a three- letter combina- 
tion of the form VCC. In this phase, there were only three guesses which 
used aN of the letters. This fact, along with the relatively short dura- 
tion of the phase, suggests that the subject was not as familiar with end- 
ings as with the beginnings of words. It Is also possible that she was 
not accustomed to constructing words "backwards." 

Phase five, which covers nodes 112-139, represents the Implementation 
of a new letter combination strategy. Here the subject used trl grams 
(three-letter combinations) primarily, making one syllable with three 
letters and then adding a fourth letter. But unlike the previous phases, 
where guesses using all six letters were made, the construction of four- 
letter combinations was more prevalent In this phase. The emphasis on 
smaller parts of solution words as seen In the last two phases suggests 
that a word recognition strategy may have operated, at times. In conjunc- 
tion with the word construction strategies of phases four and five. If 
at the end of phase three the subject had exhausted all of the possibilities 
for familiar words, she may have begun to search memory using the avail- 
able word fragments she generated as a cue to find a word (recognize it) 
that used the letters In the anagram. This recognition strategy Is simi- 
lar to strategy one. 

The last phase of the protocol, covering nodes 140-162, does not re- 
present a new strategy but a return to phase two where the subject was con- 
structing words from consonant digram beginnings. The return to a pre- 
viously used strategy caused repetition of several of the guesses (TRESOP 
at nodes 42 and 146, SPORT at nodes 21 and 162, and TROSEP at nodes 37 
and 147). At this point, the subject was stopped because she seemed to 
be frustrated and exhibited no new behavior. 

In summary, the protocol can be segmented Into six phases, repre- 
senting five different strategies^ 
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strategy I . Shallow memory search for recognizable words . Two- and 
three- letter combinations were generated as a cue with which to search 
memory for a recognizable word. 

Strategy II. Construction .of words' using CC beginnings . This strategy 
represented an exhaustive search using the pattern CCVCVC as a guide to 
the construction of words from all likely two-consonant digrams. 

Strategy III. Construction of words using CVC beginnings . Words 
were constructed according to the pattern CVCCVC of CVCVCC using consonant- 
vowel combinations, rather than two consonant digrams as In strategy two. 

Strategy IV. Construction of words using word endings . Words were 
constructed from ending to beginning using mainly two-consonant digrams 
as possible word endings. 

Strategy V. Construction of words using 5- letter digrams . Three 
■r letter combinations were generated to which a fourth letter was often 

added. Construction was less orderly than In strategy two or three and 
involved both word endings and beginnings. 

Operators . The next step in the analysis consisted of determining 
a set of operators that could account for the progress from one node of 
the PBG to the next. Four operators are defined to specify the processes 
that were used to generate a new state of knowledge using the previous 
state (previous node of the PBG) as input to the operator. Only four 
operators; are needed to account for all but a few transitions on the PBG.^ 
The operators are specified In the form OPE(A) or OPE(AB...) where the 
argument A(or AB, etc.) is the letter or combination of letters resulting 
from the^ operation. The operators Involve processes whereby a letter or 
a letter combination is generated, added, deleted, or exchanged. 

GEN(AB). First it was necessary to have some sort of operation to 
initiate each search path. This operation consisted of generating letters 
or letter combinations that were used as starting points for the construc- 
tion of words. Most of the work done by the subject In the first three 
phases- consisted of the generation of a two-letter combination as a be- 
ginning for the word* In phase two, the pairs generated were entirely 
consonant pairs, whereas in the third phase, either single letters or pairs 
of letters of the form consonant-vowel were generated. In the analysis, 
this operation is referred to as GN(AB), where the argument AB is the 
combination of letters actually generated by the subject. An example Is 
node one, which is reached by the operation GN(PR). Similarly, the 
. progression from node three to node four Involves the operation GN(TR). 

AD(AB). A second operation used frequently In the construction of 
solution words is addition. After the generation of a starting pair of 
letters, the subject added letters, either one, two, three, or four at 
a time to form a possible solution. This operation is designated as 
AD(AB). An example of this operation Js the progression from node one 
to node two that used the operation AD(0), where the letter "0" was added 



The exceptions occur where the subject made a "wild" guess or In- 
tuitively changed a guess before realizing theit it violated constraints. 
For example, at node 87 the subject went from S to SEPARATE. At node 25 
the transition was SPEROT to SPIRIT; here the "E"s were replaced by "l"s, 
probably on the basis of a sound cue. The four operators could be extended 
to account for these kinds of transitions as well, but the processing is 
probably of a different kind involving strong memory associations to the 
current information in short-term memory. 



ERIC 



14 



to PR to form the trigram PRO. The generation and addition operators 
account for most of the attempts at construction of words in the protocol. 

DEL(AB). The next operation is one by which the subject was able to 
modify guesses, reduce guesses, and reject certain combinations of letters. 
The operation accounted for actions where only a part of a word was re- 
jected and the other part retained for further exploration. The operation 
Is designated as DEL(AB), One example of such a reduction is the pro- 
gression from TRESPOR at node 54 to TES at 55. The notation for this be- 
havior is DEL(POR), because the letters POR were deieted from the solu- 
tion attempt. Another example of this operation is the move from node 
three to node four where the word PROTE Is reduced to TR. According to 
the definition of the operations, this behavior would have to be expressed 
as DEKPROTE) followed by GNCTR). This particular example was Introduced 
for the purpose of uncovering a problem with such a system of coding. 
The problem occurs because PROTES Is used again In node 12, very soon 
after (ts original rejection. It seems therefore, that the subject may 
not have actually deleted PROTE from her memory, but merely have aban- 
doned It only temporarily for another possibility. In such instances, 
the operation Is coded as a new generation, e.g., GN{TR); and the delete 
operation Is omitted, allowing for the possibility of a return to a pre- 
viously abandoned state of knowledge. 

EXC(AB). The next operation to be defined deals with manipulations , 
used by the subject once a guess Is formed, where the next move was not 
an addition or reduction, but rather some Kind of rearrangement of the 
letters. Most of the manipulations involved two letters and assumed the 
form of some sort of exchange. For example, in moving from node 103 to 
node 104, the subject exchanged the S with the P, changing POSERT to 
SOPERT. The code for this move is EXC(SP), where the argument again re- 
presents the actual letters exchanged In the move. Another ei?ample is 
the progression from node 31 to node 32, which Involved the exchange of 
the vowels. Node 31 is STEPOR and node 32 is STOPER, and the exchange 
Is coded as EXC(OE>. One Interesting observation is that certain pro- 
gressions on the graph can be represented as more than one exchange. 
For instance, the transition between nodes 65 and 66, from responses 
REPOST to RETOPS, was interpreted as two successive applications of the 
exchange operation. The move is expressed as EXC(PT) and EXC(SP), where 
the first exchange results In the Intermediate position RETOSP that does 
not appear In the protocol. The final exchange results In RETOPS. The 
exchange operator is also used to cover the very few situations in which 
the position of a single letter Is changed within a word. For Instance, 
in moving from node 79 to node 80, the subject merely changed the posi- 
tion of the T. This action Is coded as EXC(T), so that an argument of 
a single letter in the exchange operation designates a simple change In 
the position of one letter. 

The four operations described the processes by which the subject 
constructed responses and manipulated letters; as such, they are a de- 
scription of the subject's competencies in using letters to construct 
words. Table I provides a description of the operators used at each 
node of the PBG, and Table 2 provides an index of operator frequency for 
each of the six phases of the protocol. 

Additional processes . Up to this point, the processes by which the 
subject decided what to do at each step have not been spelled out in great 
detail. An additional set of processes is needed, defined at a similar 
level of specificity as the operators, to indicate how goals are set, 
hoW states of Knowledge are evaluated, and how operators are selected 
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TABLE I 



OPERATORS USED AT EACH NODE OF THE 
PROBLEM BEHAVIOR GRAPH 



1 


GN(PR) 


34 


GN(SR) 


67 


DEL(T0PS),AD(S) 


2 


AD(0) 


35 


GN(TR) 


68 


EXC(SP) 


3 


AD(TE) 


36 


AS(OS) 


69 


AD(TOS) 
EXC(ST) 
EXC(SP) ^ 


• 4 


GN(TR) 


37 


AD<tP) 


70 


5 


AD(E) 


38 


EXC(PS) 


71 


6 


DEL(E),AD{S) 


39 


OEL(OPES) 


72 


DEL(OT) 


7 


AD (OPE) 


40 


AD(OE) 


73 


DEL(SP) 


8 


GN(PR) 


41 


DEL(OE) 


74 


DEL(E),AD(0P) 


9 


Repeat #7 


42 


AD(ESOP) 


75 


AD (SET) 


10 


EXC(S) 


43 


DEL(ESOP) 


76 


EXC(PT) 


1 1 


AD(0) to node 8 


44 


GN(ST) 


77 


DXC(TP) 


1 2 


ADCTES) 


45 


GN(PR) 


78 


DEL(SET) ,AD(E) 


13 


DEL (ROTES) 


46 


AD(E) 


79 


AD(ST) 


14 


GN(ST) 


47 


AD(ST) 


80 


EXC(T) 


15 


AD(0) 


A n 

48 


DEL(EST),AD(0) 


81 


GN(PROS) 


16 


AD (PER) 


49 


AD (SET) 


82 


GN(S) 


17 


GN(P) 


50 


EXC(OE),EXC(TS) 


83 


AD (TROPE) 


18 


GN(POS) 


51 


GN(T) 


84 


GN(REP) 


19 


AD(TER) 


52 


AD(ES) 


85 


GN(TER) 


20 


GN(SP) 


53 


AD(P) 


86 


AD(S) 


21 


AD (CRT) 


54 


AD (OP) 


87 


GN(S) \ 


22 


AD{e) 


55 


DEL(POR) 


*88 




23 


DEL(TER),AD(T) 


56 


GN(ST) 


89 


AD(EPROT) 


24 


AD(ER) 


57 


AD(PE,ER) 


90 


DEL(PROT),AD{T) 


25 


EXC(OT,ER) 


58 


GN(PER) 


91 


EXC(TP) 


26 


AO(OT) 


59 


AD (SOTS) 


92 


GN(PRO) 


*27 




60 


GN(RE) 


93 


AD(SEP) 


28 


GN(ST) 


6i 


AD(SP) 


^94 




29 


AD(ER) \ 


62 


AD(OT) 


95 


DEL(OSEP) 


\ 30 


AD (OP) 


63 


EXC(PT) 


96 


GN(SET) 


31 


Cexc(r,p) 


64 


DEL(STOP),AD(P) 


97 


AD(ORP) 


32 


EXC(0,E) 


65 


AD(OST) 

EXC{PT),EXC(SP)^ 


98 


DEL(SETO) 


33 


DEL(OPER) 


66 


99 


AD(E) 
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TABLE 



I (cont.) 



100 


AD(T) 


101 


GN(RT) 


102 


AD(E) 


103 


AD(POS) 


104 


EXC(SP) 


105 


GN(PERS) 


106 


GN(ERT) 


107 


EXC(TP) 


108 


EXC(OE) from 


109 


AD(PtRS) 


1 10 


EXC(SP) 


*l 1 1 




1 12 


DEL(OE) 


113 


GN(E) 


1 14 


GN(OPREST) 


115 


DEL(OT) 


1 16 


DEL(ES),AD(0) 


1 17 


DEL(P) from 1 


1 i8 


EXC(SP) 


119 


GN(ROSE) - 


120 


AS(PT) 


121 


GN(PREST) 


122 


AD(0) 


123 


DEL(OT) 


124 


EXC(OE) 


125 


EXC(EO) 


*I26 




127 


DEL(S) 


128 


AD(S) 


129 


DEL(E) 


130 


GN(E) 


131 


AD(OT) 


*See 


footnote 6 



132 


EXC(PT) 


133 


AD(SPR) to 131 


134 


DEL(S) 


135 


DEL(OTE) 


136 


AD(S) 


137 


DEL(S) 


138 


AD(T) 


139 


EXC(RP) In 137 


140 


EXC(PR) 


141 


AD(OTES) 


142 


EXC(ST) 


143 


EXC(OE) 


144 


DEL(SOT) 


145 


EXC(PT'),AD(OS) 


146 


EXC(SP) 


147 


EXC(OE) 


148 


EXC(PS) 


149 


GN(ST) 


150, 


EXC(S) In 148 


151 


DEL(RO) 


152, 


AD (OR) 


153 


GN(PORTES) 


154 


Return to 150 


155 


EXC(OR),EXC(PE) 


*I56 




157 


EXC(EP),EXC(OR) 


158 


GN(PRO) 


l'59 


eXC(OR) from 157 


160 


EXC(OE) 


161 


GN{SP) 


162 


AD(ORT) 
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TABLE 2 

OPERATOR FREQUENCY IN EACH OF THE 
Six PHASES OF THE PROTOCOL 



Operation 






Phase 


of 


Protoco 1 








1 


1 1 


1 1 1 


IV 


V 


VI 


Total 


GN 


6 


6 


10 


3 


5 


4 


34 


AD 


10 


15 


21 


5 


8 


4 


63 


EXC 


1 


6 


10 


4 


5 


13 


39 


DEL 


2 


5 


9 


1 


10 


2 


29 


TOTAL 


19 


32 


50 


13 


28 


23 


162 
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and applied,. As our interests were centered on the general structure of 
problem solving ("higher order" strategies and basic operators) a detailed 
analysis of all decision processes was not attempted. These kinds of pro- 
cesses were, however, not completely neglected In our analysis. Many of 
these processes are indicated or implied in the previous discussion of 
heuristics and strategies and in the summary model to follow (for example, 
see figures I and 2 where several subroutines in the flow charts of Stra- 
tegies I and II indicate decision processes).' 

A first order model of anagram problem-solving . Our analysis of the 
protocol is now completed. To integrate the presentation to this point, 
a first approximation to an information-processing n>odel is outlined 
based on the strategies and operators defined In the analysis. The model 
concentrates on those phases of the protocol that exhit)ited a fair amount 
of regularity, and principally on Strategies II and III. The models for 
these striategies are presented as flow charts !n figures I and 2. 

Processing proceeds as follows. First, working from left to right, 
the first goal is defined, which In this case Is a word fitting a partic- 
ular pattern (CCVCVC). With this pattern as the goal, the first step in 
the construction of words is the application of the rule-out factor to 
eliminate unlikely word beginnings (the beginnings are unlikely, simply 
because thdy do not occur in the English language as the beginnings of 
words). Since there are four consonants and two vowels, a decision is 
made that two consonants would have to go together. Consonant pairs are 
considered first, and unlikely pairs are ruled out. At this point, 
vowels are also ruled out as occurring In the first or last position of 
the solution word.^ 

It should be noted that at each point In the solution process In 
which all of the letters are used for the formation of a guess, memory 
is scanned to see if a familiar word has been constructed. If a match 
(the word constructed and a word In memory being identical) Is made, the 
entire solution is terminated successfully. If no match is made, the 
process is continued. 

The next step Is the generation of a starting consonant pair that 
was not previously ruled out. After the initial generation, another pair 
of letters is added. This second digram Is of the form VC, forming a 
four-letter string of the form CCVC. After the first addition, a simi- 
lar second addition Is made by adding the remaining two letters to form 
a possible solution word. An evaluative process Is then applied that 



In this respect, it can be said that, we are folloy/ing a standard 
programming convention in attempting first to describe general problem 
Solving structure leaving some details as subroutine calls, with the sub- 
routines to be detailed later after the general structure Is completed. 
Newell and Simon (1972) provide an extensive discussion of the kinds of 
decision processes that are needed here, and are generally found to occur 
in problem solving and, In many existing problem-solving programs (see 
especially Chapter 4, pages 101-105, and Chapter 5, pages 191-203). 

8lf a vowel were used in the first or last position, only one vowel 
would be left to break up four consonants. The unlikely pattern (CCVCXJV) 
was correct, however^ for the second solution PRESTO. Thus, ruling out 
a vowel as the last letter may be the principle reason that CCV(XV was 
never a goal pattern and the subject never reached PRESTO. 

I 

19 



I 



YES 



MORE 

:ONSONANTS^ 



SET A PATTERN 
AS A GOAL 
(CCVCVC) 




SELECT 
DIFFERENT 
STRATEGY 



RULE OUT UN- 
LIKELY CONSONANT 
(CO BEGINNINGS 




GENERATE CC PAIR 
AND AOD VOWEL 
AND CONSONANT 
TO FORM CCVC 



ADD VOWEL AND 
CONSONANT TO 
FORM CCVCVC 



REJECT 
STRUCTURE 



SUCCESS 
END 



3 



EXCHANGE 
LETTERS 



Fig. I. Flow chart of Strategy II ( Phase n ) 
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DEFINE COAL 
(CVCCVC) 

~~r~ 

RULE OUT ANY 
UNLIKELY COM- 
BINATIONS 




NO 



Fig 2 Flow chart of Strotegy m (Phase IE) 
2t 

ERLC 



results in a decision to continue with the same basic structure and make 
exchanges of letters. This point represents a two-branch decision point. 
If the decision is to continue with the same structure, exchanges are per- 
formed using primarTiy the last four letters of the word, keeping the ini- 
tial digram Intact. The exchanges consisted of either exchanging the 
vowels or exchanging the consonants in positions four and six. Assuming 
that the exchanges of letters are performed and no match results, some or 
all of the letters are deleted. If another available untested consonant 
pair is present, it is generated and the above procedure is repeated. 
If at the decis5on point the decision is not to continue with the same 
basic structure and make exchanges of letters, the next action will be the 
same as if all manipulations had been attempted. Another digram will be 
generated and the construction process initiated once again. 

When the point arrives where all likely consonant pairs are exhausted 
and no match has yet been made, the goal pattern used initially is aban- 
doned in favor of a secondary pattern CVCXiVC. This is the point of tran- 
sition from phase two to phase three. In phase three, the generations are 
less precise; sometimes a single letter is generated Initially and two 
letters added, and sometimes three letters are generated. Occasionally, 
a digram is generated and a single letter added. The generations in this 
phase usually result in a three-letter combination of the form CVC. With 
this segment as the root, three other letters are added using the same 
pattern, so that the final word has the form CVCCVC. If at this point no 
known word has bean constructed, an evaluative prcc-^ss is applied in order 
to determine whether or not exchanges should be made. Depending on the 
resultant decision, the word Is either manipulated or reduced as explained 
above. When a "dead end" is reached, the guess is again reduced and an- 
other generation attempted^ 

This step by step description of the behavior of phases two, three, 
and six (same as phaso II) can be easily adapted to simulate the behavior 
of phase four, where the subject briefly used endings of words as build-* 
Ing blocks. The adjustment is in the generation operation where likely 
ending digrams rather than beginning digrams are generated, and the con- 
struction of words proceeds from right to left. The same inner manipu- 
lations are performed. Reductions are applied to the first part of the 
word, leaving the ending as tha root. 

General Discussion; Experiment I 

The problem-solving behavior of a single subject was analyzed into 
the basic operators and strategies used in solving a single anagram, and 
a preliminary Information-processing model was constructed based on the 
analysis. There are several points that should be considered In rela- 
tion to this analysis. First, the analysis is based on a single proto- 
col and no claims for assessment of Its generality can be made at this 
point (this topic Is taken up In Experiment Two). 

There are also certain occurrences In the protocol that have not 
been adequately described, principally the detailed structure of the de- 
cision processes that guided the subject's basic processing (see footnote 
5). Several observations about these processes, however, are suggested 
by aspects of the protocol and the experimental information on anagram 
problem-solving. One place where an evaluative process Is Implied \l In 
those situations where a guess was abandoned before all the possible 
manipulations of the letters In the guess was attempted. For example. 
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at node 54, TESPOR is reduced without going on to some of the exchanges 
that are possible at that point and were performed at other similar junc- 
tures. In some situations the subject performed manipulations such as ex- 
changes, white in others the guess was reduced before attempting any such 
exchanges. One possibility Is that the subject did in fact make exchanges, 
but they were hot verbalized. A more likely possibility is that an evalua- 
tion of the potential value (In terms of yielding the solution word) of 
transformations was made, and a decision reached to abandon the present 
line of development for another letter combination as a starting point. 
Experimental research on anagrams suggests that theie decision processes 
for evaluating fruitful lines of attack, and thus in selecting operators, 
is based on the subject's knowledge of certain propeV'ties of English, par- 
ticularly digram frequency, letter position frequency (the probability 
that a certain letter will appear more frequently in one position than 
another in a word), and letter sequence Information (Mayzner & Tresseit, 
1962; Mayzner, Tresseit, & He I bock, 1964; Johnson, 1965; Dominowski, 
1968). Evaluative processes based on such information can be viewed as 
extensions of the rule-out factor discussed previously, in that unlikely 
possibilities, whether of digrams, letter position, or order, are "ruled- 
ouf ; and when the likely possibilities have been exhausted a line of 
search fs terminated.^ 

Another feature that may have guided search is 1he sound of a letter 
combination. The auditory trace of the sound of a letter combination may 
remain in acoustic storage and be used in an adjacent operation. For 
example, in the construction of the first correct solution POSTER, the 
subject had three nodes earlier formed the word STOPER, which has the 
same digram ending and the same long sounding "0" at the beginning of 
the word I^STER. The solution process employed by the subject may well 
have included the auditory trace of the sound -0-ER, and this information 
may have been used together with the generation of I^S (at node 18) to 
form POSTER. 

Experime nt II; Discovery of Additional Problem-Solving Strategies, 
and Generalizations of a Preliminary Model or Anagram 
Problem-^Solving to Several Subjects ~ 

The present experiment is an attempt to extend the account of prob- 
lem-solving mechanisms developed In Experiment I to several subjects. 
The behavior of several subjects on the same aniigram RETOPS and a new 
anagram TAEMG was analyzed for evidence of the subjects' problem-solving 
strategies and operators. New strategies were described and compared 
with the strategies formulated in Experiment I, and the preliminary model 
of Experiment I was revised to incorporate these additional processes 
and finally, a computer simulation of the model was developed. 

^The suggestion that decision processes involve the problem solver*s 
knowledge of the properties of English usage, implies that search paths 
In the PBG could be analyzed In terms of language variables as an aid 
In the specification of decision processes. The suggestion also indi- 
cates the need for a closer integration of the information processing 
and experimental approaches to studying cognitive phenomena. 
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Method 



Apparatus. The apparatus consisted of two sets of six cardboard 
blocks representing two experimental anagrams. Each block measured 
approximately one inch square and had one letter printed on one side, 
A casette recorder was used to record the subject's behavior. 

Procedure . The subjects were tested individual Each subject was 
asked to read a set of typed instructions explaining the nature of the 
task. The instructions designated the purpose of the experiment to be the 
analysis of the i nd iv ?dua i ^s approach to the probiem. Including strategies 
and specific manipulations used in attempting to solve the problem, rather 
than solution time. The subjects were asked to "think-aioud," that is, 
to comment on everything considered while attempting to solve the problem 
and to give reasons whenever possible for their behavior. 

After the subjects finished reading the instructions, the experimen- 
ter re-emphasized the importance of thinking atoud and answered any ques- 
tions. Each subject was then presented with the two anagrams, one at a 
time, RTEOPS {solution words POSTER and PRESTO) and TAEMNG (solution word 
MAGNET) in the form of the movable cardboard blocks. No time limit was 
imposed, and the subject worked until a solution was found, or until he 
asked to be told the answer. 

The verbalizations were tape recorded, and at the same time a written 
protocol was recorded by the experimenter. The written protocol was a 
list of the rearrangements of letters made by the subject. 

Subjects . The subjects were seven undergraduate students enrolled 
in Introductory Psychology courses at Michigan State University; five 
were female, two male. 

A na lysis . The written protocols and the tape recordings were ana- 
lyzed by procedures similar to those described in Experiment I, for evi- 
dence of the subjects' strategies and operators. These processes were 
then compared with the strategies and operations described in Experiment I 
to determine how well the original mechanisms accounted for the behavior 
of the new group of subjects. 
( 

Results and Discussion 

1 

\Strategy and operator use . Table 3 presents a breakdown of the 
strategies used by the subjects. As expected, the previous identified 
strategies were well represented in the behavior of the subjects. Stra- 
tegies II and 111, which accounted for the largest share of the process- 
ing in Experiment I, again appear to be major strategies. Only these two 
strategies were used by all the subjects. New processes were used by 
three of the subjects in addition to the other strategies; however, no 
new strategies were used exclusively by any of the subjects. 

Table 4 gives a breakdown of the operators used by the subjects. 
No new operators were identified; however, the pattern of operator usage 
is quite d if ferent than was found previously. AM of the subjects used 
the Generation and Addition operators, but the selection operator was 
employed by only one subject, and two subjects failed to make any use of 
Exchanges. 

Processing by individual subjects . Individual differences In problem- 
solving procedures existed among the subjects, and none of the subjects 
showed the same degree of orderly transition between strategies and the 
systematic use of operators as the subject ot Experiment I* (This may 
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TABLE 3 

PROBLEM-SOLVING STRATEGIES USED BY EACH SUBJECT 





S+r»tfinlp«; 
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1 


II 1 1 iT 1 

(from exp. 


IV 
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New 

Strategies 


# Strategies 
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X 


X 


X 






X 


4 


2 




X 


X 








2 


3 


X 


X 


X 




X 




4 
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X 


X 


X 


X 
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5 


X 


X 


X 


X 


X 


X 


6 


6 


X 


X 


X 


X 


X 


X 


6 


7 




X 


X 




X 






# of Ss 
Using each 
Strategy 


5 


7 


7 


3 


5 


3' 


JT = 4.3 



TABLE 4 

OPERATIONS USED BY EACH SUBJECT 



Subject 



GN 



Operators 



AD 



Del 



EXC 



Other 



I 

2 
3 
4 
5 
6 
7 



X 
X 
X 
X 
X 
X 
X 



X 
X 
X 
X 
X 
X 
X 



X 
X 
X 
X 
X 



represent Inadequacies in the data, rather than an accurate description 
of processing. None of the subjects verbalized as well as the original 
subject, and there were more gaps In the protocols of these subjects.) 
One common feature of processing, however, was the use of rule-out mechan- 
isms; all of the subjects employed the "rule-out factor" at some point 
in their attempt to solve the anagrams. 
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The main features of each subject's processing is briefly described 
below. 

Subject i . The first subject made use of a new strategy, which rep- 
resented a major portion of his behavior and, which can be characterized 
by the following algorithm: GN(2), AD(2), Search Memory. One or two 
letter combinations that form a recognizable sound are generated, and 
this group of letters is used as a cue with which to search memory for a 
word having that particular sound. If a word Is found. It Is broken up 
into its component letters and checked against the anagram. The general 
strategy, whereby a word Is first accessed and then checked against the 
letters of the anagram has been previously suggested as a strategy for 
solving anagrams, and termed a ''sol ut ion-backward" method (Moira, 1971). 
Sound features, however, have not been previously hypothesized as cues 
for memory search in looking for sulution words* 

S ubject 2 . This subject made almost exclusive use of Strategies II 
and III, although there was no orderly progression from one strategy to 
the next. 

Subject 5 . This subject used several of the construction strategies, 
but his solution attempts for the first anagram (RETOPS) consisted mainly 
of two digrams ST and PR. More than 50% of his responses contained one 
or both of these bigrams. 

Subject 4 . This subject exhibited a wide variety of behavior in- 
cluding brief use of each of the five original strategies. The major 
difference was the lack of use of the Delete operation. The subject, 
rather than retain parts of words and reduce them, would construct 
words from entirely new beginnings. 

Subjects 5 & 6 . Both these subjects exhibited all the original 
strategies, although the transitions between strategies were not orderly. 
Both subjects also used the new solution-backward strategy. 

Subject 7 > The seventh subject, like subject two, made primary use 
of the construction strategies. Strategies II and III. This subject was 
the most similar of the seven to the original subject, both in the choice 
of strategies and in clearer transitions between strategies. 

Revisions in the preliminary model of problem solving . As a result 
of Experiment II, new information was gained about the processes subjects 
use In solving anagrams. This information was used to make several addi- 
tions to and changes in the first model. Originally, the construction of 
words using letters from the anagram was given primary emphasis. The re- 
sults of the present experiment confirm the Importance of construction 
strategies. But, in most of the subjects tested In Experiment II, con- 
struction was not well organized; rather than an orderly progression from 
one strategy to the next, subjects employed two or three strategies al- 
ternately switching back and forth between them. The switching Is evi- 
denced by the apparent lack of use of the Delete operation. Instead of 
reducing some part of a response, the subject would shift his attention 
and retrieve a different strategy and start the construction process over. 

The new solution-backward strategy Is also more Important than re- 
cognized previously. In the original analysis, word generation strate- 
gies were mentioned only briefly, because they were evident In Jess than 
five percent of the subjects' protocol • Since the preliminary model was 
composed of two distinct construction algorithms (for Strategies II & III) 
using two different goal patterns, with no interconnection between the al- 
gorithms, the Inclusion of the working backwards strategy in the model 
simply entails adding the algorithm for this strategy to the two existing 
algorithms. A flow chart for this algorithm appears in Figure 3. 
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SOUND 



rig. 3. Flow chart of "solution-backward" strategy using recognizable sounds 
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A subject who employed this strategy would first looK at the ana- 
gram in search of a recognizable sound, that is, a combination of two 
or three letters, vowels or consonants. If a recognizable sound did not 
appear, the subject would change strategies, probably to a construction 
technique. However, if a recognizable sound was available, the subject 
searched memory using the first sound as a cue to a word. If a second, 
as we! I as a first sound was aval lable, the cue used would be a combina- 
tion of the two sounds. In either case memory was searched usually un- 
til a word was found. The word was then broken into Its component letters 
and each letter was tested against the letters in the anagram, if all 
the letters matched, a solution was found. If not, the process was re- 
peated . 

The preliminary model also could not account for the order in which 
digrams were generated; that is, it could not account for, for instance, 
why the subject initially generated PR rather than ST, the most frequently 
occurring bigram in positions one and two. The analysis of the subjects' 
behavior in the present experiment Indicates that the original order of 
letters in the anagram influenced the order In which digrams were gener- 
ated. For example, the letter in position one was more likely to be used 
to find a starting digram than the letter in position six. 

A computer simulation of the model . The revised model of anagram 
problem-solving was implemented on the CDC 3600 computer at Michigan 
State University using the HINT Programming language. HINT (Hierar- 
chical Information Nets) is a graph processing language designed pri- 
marily to extend list processing semantics to graph processing (Hart, 1969). 
The simulation (and the model) is not intended as a detailed representation 
of the mechanisms of a single subject, but rather is intended as a simu- 
lation in general of some of the mechanisms people use to solve anagrams. 

Features of the program model other than strategy and operator rou- 
tines, are the data structures including a model of memory. The memory 
is constructed along lines suggested by Frijda (1970, 1972) and Under- 
wood (1969). Underwood suggested that i nformation storage could be 
viewed as a col lection of attributes, and Frijda suggested that the in- 
formation store may consist of a relational network of labeled links. 
Both of these ideas are incorporated In the model of memory. The infor- 
mation stores are labeled graphs with each node consisting of a set of 
attributes. The nodes are connected by either Associative or Construc- 
tion links, or both. Associative links are based mainly on acoustical 
association, and the Inclusion of such a type of link is a reflection of 
the large amount of attention given to the auditory recoding of informa- 
tion in memory (Norman, 1969), Words linked together by Construction 
links are words with similar roots, a reflection of their orthographic 
similarity. According to Underwood (1969) orthography Is one of the attri- 
butes by which information can be referenced. 

Each node in memory is represented as a set of attributes. For words, 
the attributes used In the present model are frequency of occurrence of 
the word In everyday usage, the spelling of the word, and the number of 
letters In the word. Other attributes specified by Underwood, such as 
spatial and temporal features, do not appear to be used in the solution 
of anagrams. Digram frequencies are represented In memory by three 
labeled graphs. Each graph has 26 nodes representing the letters of the 
alphabet. Directed labeled links connect the nodes, the labels being the 
frequency of occurrence of the particular digram based on counts made by 
Mayzner and Tresselt (1965). Three graphs were used to differentiate 
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between the occurrence of digrams in different positions In six-letter 
words. Of the three graphs, one represents digrams appearing in posi- 
tions 1-2, one represents the digrams in positions 3-4, and the last rep- 
resents those In positions 5-6. More than one graph was included be- 
cause it was assumed that a subject solving an anagram could, for example, 
recognize that ER Is more commonly a word ending than a word beginning, 
and that PR occurs more often In positions 1-2 than In 3-4 or 5-6. A 
single graph could not have differentiated between fJI grams appearing In 
different positions. These three graphs together attempt to simulate the 
same features of subjects' knowledge of the English langauge that is 
heavily relied upon in the solution of anagrams. All words are repre- 
sented as lists. For instance, the word constructed by the program has. 
the variable name GUESS, and if the guess were the solution word POSTER 
the representation would be GUESS=(P,0,S,T,E,R) , where the order is sig- 
nificant. List processing operations are used to insert and delete ele- 
ments from lists, that is, to construct or reduce guesses. 

The first consideration of the program's problem-solving behavior 
is the rule-out factor. The rule-out factor is actually built into the 
digram frequency graphs. Since a digram that never occurs In the English 
language In the position considered by the graph has no representation 
on the graph. It can never be generated, and has therefore effectively 
been ruled out. 

Each of the four operators discussed in the presentation of the model 
is represented by a subroutine. The first subroutine Is GENERATE. It 
was mentioned that the order of generation of digrams has some relation- 
ship to letter position in the original anagram, as well as to the fre- 
quency of occurrence of the digram In English. In the model, one gener- 
ation subroutine generates the first digram that exists on a graph. The 
nature of the data structure in HINT is such that when the graph has been 
constructed from the anagram, the order of letters in the anagram Is re- 
tained, and the generation of the first digram will reflect the original 
order . 

The other subroutines which represent the operators are ADD, REJECT, 
and EXCHANGE. These all use list process! ng. 

The operation subroutines are also used to form a second level of 
subroutines that represent the strategies outlined in" the present model. 
The two construction routines are relatively straight forward, using 
different combinations of the operators. The third routine that simu- 
lates the new memory search strategy starts with the generation of a 
b I gram. All words beginning with this b I gram are tagged, and the most 
frequent one Is generated. Next, the closest linked node is gensrateck 
simulating association. The process Is repeated until either a solution 
is found or the possible cues are exhausted. \ 

General Discussion: Experiment I I 

Several inadequacies exist In the present program and help to point 
out some of the gaps In our Information processing analysis of problem- 
solving behavior. The program, as well as the model, lacks an effective 
executive to direct execution of the strategies. There Is also no 
switching behavior at this point. This reflects the difficulties In 
determining why a subject used a particular strategy when he did, and why 
a subject switched strategies when he did. An effective executive must 
be developed to control the execution of particular strategies. At this 
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point, all of the strategies are exhaustive. \For example, alt blgrams 
are generated before a switch of strategy is made, whereas the subjects 
switched many times and returned to strategies during the course of their 
solution attempts. A complete model requires the formulation of the de- 
cision processes, which guide p'rociassing (see the discussion in Exp. I). 

The model is also limited in that it was designed to deal only with 
a single anagram (RETOPS). This is more of a limitation of programming 
rather than the model. For example: The vocabulary store was limited to 
only the words generated by the experimental subjects, and the digram fre- 
quency graphs were designed for words of only six letters. Revisions of 
the model to include a large vocabulary store, or to include digram fre- 
quency information for words of different lengths, do not represent changes 
In the process or representations of the model; but rather, would allow 
the model to handle a wider range of anagrams using the same processing 
mechanisms as before. 

While our analysis and model is admittedly incomplete and narrow in 
several respects the programmed model has served one major purpose: To 
test the adequacy oi the descriptions given for the strategies and opera- 
tors of Experiments I and II. That the mechanisms could be embodied as 
the central processes in a running computer program that can solve ana* 
grams of the kind used In the experiments, suggests that the processes 
constitute adequate theoretical descriptions of some of the principle 
psychological processes employed by persons when attempting to solve 
anagrams . 
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Appendix I Chapter 2 
Protocol of Subject One on Anagram RETOPS 



SUBJECT I 



ANAGRAM #1 



RTEOPS 

P-R 

PR 

PRO 

PROTE 

pause 

TR 

TRE 

pause 

S-T-R 

Phase I STROPE 

pause, urn 
PR 

STROPE 
pause 
TROPES 
PRO 

PROTES 
P 

S-T 
STO 

S TOPER 
pause 

maybe it ends in P 
PCS 

node 20, POSTER (first solution 
PBG obtained) 



S-P 

SPORT 

S PORTE 

pause 

SPOT 

SPOTER 

SPER 

SPEROT 

SPIRIT 

pause 

ST 

S-T 

STER 

STEROP 

STEPOR 

pause 

STO-PER 

ST 

try S-R 

SR doesn't go together 

T-R 

TROS 



TROS EP 

TROPES 

TR 

TROE 

OE doesn't go together 

TRESOP 

TR — SP 

I had ST a I ready 

try PR 

P-R Phase I I 

PRE 

PRE ST 

PRO 

PROSET 

PRETOS node 50 PBG 
pause 

I don't think there Is another 
word, okay, wait 

T 

TES 
P 

TES PGR 
TESPOR 
TES 
S-T 

what goes together? 

what letters? okay 

PESTER 

pause 

PER 

PERSOPS 
pause 
okay 
RE 

RFSP 

RESPOT 

RESTOP 

pause 

REP 

REP 

RE POST 

REPOST 

RETOPS 

RES 

REP 

REPTOS 

REPSOT 

RESPOT 

no, that's respite, um 
RESP 



ERIC 
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SUBJECT I ANAGRAM #1 (cont'd) 



ERIC 



R-E 

pause 
ROP 

let's try this 
ROP 

ROPSET 
ROTSEP 
ROPSET 
pause 

there's not another word I 
um 

ROPE 

ROPEST 

TROPES 

PROS 

PROS 

There's one too many 

letters here! 
PROS 
S 

STROPE 

that's not a word 
um 
REP 
TER 
TER 
TERS 
Phase I I I pause 

okay, I'll start with S 
cause that never goes 
in the right place 

S 

SEPARATE 

S-E-P-R-O-T 

um, pause 

SET 

SEP 

PRO 

PROSEP 

there needs to be another T 

It's PROTEST 

PR 

SET 

SETORP Node 97 PB6 

R-P 

ERP 

TERP 

t don't think those — 

R-T 

-E-R-T 

e*r*t 

Phase IV That's a good one 
POSERT 
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SOPERT 

Guess that's not such a good one 

PERS 

ERT 

ERP 

ORT 

try this one 
ORT 

PERSORT 
SERP— 

SUPPORT Node III PBG 

I swear there's not another word 
in here. Gotta think of a. letter 
combination and I can't think — 
0 and E don't go together so 
they gotta be separated. But 
maybe it starts with a vowel. 

E 

pause 

OPREST 

PRES 

PRO 

PRO 

RES 

REP 

ROSE 

PTROSE 

um, try this again 

try something again 

PREST 

OPREST 

um 

PRES 
PROS 

PRES Phase V 

PRIEST 

That should be an I 

um 

PRE 

SPRE 

S~P-R 

oh, maybe it ends in E 

OTE 

OPE 

SPROTE 
um, pause 
PROTE 
PR- 
okay, gotta go through letter 

combinations. 
SPR goes together, but nothing 

comes after that 
pause 



SUBJECT I ANAGRAM #1 (cont'd) 



PR 

T-P-R doesn't 

R-p Node 139 PBG 

okay, let's try P-R 

P-R-O-T-E-S 

PROTES 

PROSET 

PRESOT 

I pause 

PRE 

urn, guess that's about it 
for that one. Let's 
try this: 
TREPOS 
TRSOP 
, TROSEP 
TROPES 
long pause 
urn, okay 
S-T 
STROPE 
ST — P 
STORP 

PORTES Phase VI 

STROPE 

STOREP 

STOREP " 

urn 

STIRRUP 

urn, pause 

STEPOR 

PRO 

STEPRO 

STOPRE 

STOREP 

I did that one already 
S-P 

SPORT Node 162 PBG 

CALLED (second solution was not found) 
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CHAPTER 3 

MEDIATIONAL PROCESSES IN PROBLEM SOLVING: 
STYLE OR STRATEGIES? 



A number of cognitive *'traits" have been postulated to help account . 
for individual differences in the way people process information. These 
constructs are given a variety of names including "cognitive styles" 
(Witkin, et al., 1954), "conceptual styles" (Kagan, Moss, & Siegel, 1963), 
and "cognitive controls" (Gardner, Jackson, 4 Messick, I960). The term 
"cognitive style," however, is generally used to refer to the group of 
cognitive traits that are assumed to represent individual consistencies 
in the way people process information across a variety of tasks and 
situations. 

There is some disagreement whether cognitive styles represent new 
cognitive traits, or are just manifestations of cognitive factors in 
mental ability that have been previously identified (Dubois & Cohen, 1970; 
Vernon, 1971). Cronbach, (1970) for example, has rejected the traditional 
personality trait conception of cognitive styles. He suggests that 
many tests used as Indicators of styles are, !n reality, mental ability 
tests, and that the dimensions measured by the tests represent common 
ability factors. In this view^ Sherman (1967) has presented a reasoned 
argument suggesting that the "f ield-independence-dependance" dimension 
of Witkin (1964) is nothing more than spatial ability. 

Another source of controversy is the claim (e.g,^ Wftkin, et al., 
1954; Witkin, 1964; Warr, 1970) that cognitive styles represent a general 
tendency to cope with information In a self-consistent way across time, 
situations, and informational domains. This assumption has also been 
criticized (Cronbach, 1970; Gruen, 1957; Postman, 1955), and empirical 
evidence foi^some styles shows a lack of generality across informational 
domains (Thdmy, 1972), and a lack of consistency in style usage across 
tasks (Davis, 1971). If substantial evidence for the generalized ex- 
pression of cognitive styles cannot be found then cognitive styles may 
not be generalized traits that represent consistently employed and pre- 
ferred modes of processing information, but may represent cognitive pro- 
cesses (strategies) that are more specific responses to the kind of task 
situations in which they are measured. 

Three cognitive style dimensions were investigated; Flexibility- 
rigidity, ref lection-lmpulsivity, and g loba I -ana lyt 1 c. As conceived in 
this study, the flexible-rigid dimension is made up of two components: 
(a) using a variety of approaches or strategies, and (b) overcoming 
perseverati ve behavior, such as trying over and over again to use an un- 
productive strategy for solving a particular problem (Leach, 1970; 
Marshall, 1971). The ref lect ion-lmpu I s i vi ty dimension is seen as a mea- 
sure of tempo and the carefulness with which people work on intellectual 
tasks (Kagan, 1965), and is concerned with the degree to which persons 
reflect on the validity of their hypotheses In problems that contain al- • 
ternative solution possibilities. People classified as reflective are 
thought of as being deliberate and cautious, not taking chances, and 
taking plenty of time as they work. The global-analytic dimension Is 
believed to represent the degree to which people reorganize stimulus 
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material presented to them. Witkin (1964) has characterized the analy- 
tic component of the style found In perceptual processing as "field- 
Independence." Persons described as global tend to leave the various 
elements in a stimulus field as they find them rather than break them 
down Into smaller units for analysis, while analytic persons tend to 
break-up and analyze a stimulus field into its elements^ 

Problem solving would seem to be an Ideal medium for investigating 
the use of different cognitive styles and strategies. For example, if 
a subject is presented with a complex problem, he may begin immediately 
with the first strategy that comes to mind (impulsiveness), or he may 
take his time and consider several strategies before beginning (re- 
flection), he may break the problem into subproblems (analysis), or look 
for the overall picture (global), stick with a hypothesis when It has 
failed (rigidity), or shift to a new hypothesis (flexibility). Problem- 
solving behavior also permits the observation of multiple aspects of 
processing. For example, a subject could display his tendency to be re- 
flective by thinking about the problem before deciding how to first 
attack it, by proceeding systematically, by taking notes, by being 
careful and cautious, and not guessing, etc. 

In this study, the three cognitive style dimensions formed the 
theoretical basis for the construction of "str^ategy" assessment ques- 
tionnaires designed to measure subjects' reported use of cognitive pro- 
cesses on verbal and numerical problem-solving tasks. To this end, 
separate but similar questionnaires were developed for each of the two 
kinds of problem-solving tasks. Each Itemvwas a short description of a 
process that a subject might have used In solving the problem. A few 
items were also written to be descriptive of subjective states that 
were hypothesized to accompany or contribute to the level of anxiety 
experienced while working on the problems. Examples of items judged by 
the investigators to represent, each dimension are as follows: (I) Rigidity — 
"I found myself trying to use the same hypothesis again and again." 
(2) Reflection — "I thought about the problem briefly before I went to 
work with the actual calculations." (3) Global style — "I tried to visual- 
ize the entire problem as a unit." (4) Anxiety state— "At times I worried 
that I might not be able to get the right answer." Each anticipated dl* 
mens ion was represented by several Items. 

The study addressed several major questions: (a) Would the theore- 
tical cognitive style-dimensions appear in a factor analytic analysis of 
the scales, or would more specific strategy dimensions summarize the sub- 
jects' problem-solving processes? (b) To what extent are subjects' cog- 
nitive strategies and styles cons 1 stent across problem domains and differ- 
ent degrees of task similarity? This question Is concerned with the de- 
gree to which subjects tend to respond to tasks by adopting the same or 
similar strategies, and the task parameters that influence strategy choice 
and changes in strategies, (c) To what extent are processing measures 
related to each other across similar and across different problem domains? 
(d) The relationship between subjects' use of styles and strategies ana 
problem^solving performance. 

Method 

Prob^lem-soi vi ng tasks . Ail subjects were asked to solve two kinds 
of deductive reasoning problems. One kind of problem was Verbal In form, 
the other Numeric. Both problems were fairly difficult; the mean solution 
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rate in this experiment was 45So. The Verbal Problems were who-done-it 
type mystery problems that contained only verbal material. Each problem 
involved a murder mystery, and both problems were adopted from a booklet 
of deductive reasoning problems (Summers, 1968), For each problem, the 
subjects were given a printed list of facts about a murder that contained 
all the information needed to deduce which of four persons was the killer, 
the victim, an accessory, etc. The problem required correctly deducing 
the roles of each of the four persons. Subjects were ^5! lowed a maximum 
of eight minutes to reach a solution. 

The Numeric Problems were math word problems that involved the use 
of numbers In some simple additions and subtractions. One of the Numeric 
Problems, The Hotel Problem, may be found in slightly different form in 
many popular quiz and puzzle books (e.g., Leeming, 1946, p,l5). The 
other Numeric Problem, The Horse Trading Problem, has been used in re- 
search on problem solving (Maier & Burke, 1967). These problems were 
read aloud to the subjects by the experimenter. For each problem, a 
series of facts that concerned different amounts of money being paid and 
received by different persons was given in story form. The subjects 
were required to determine how much money ended up in the possession of 
persons named In the problem, and to name which of five choices was the 
correct response. Four minutes was allowed for the completion of each 
Numeric Problem. 

Strategy-assessment questfonnal re . Two different questionnaires 
were constructed, one for each of the^two kinds of problem-solving tasks, 
The three cognitive style dimensions mentioned in the introduction formed 
the theoretical basis for item writing. A few items were also developed 
to measure subjective states that could accompany or contribute to the 
level of anxiety experienced on the problem-solving tasks. For the style 
dimensions, each item was a one-sentence description of a cognitive pro- 
cess judged to represent one of the style dimensions that the subject 
might have employed while working on the problems. Each statement was 
written to indicate what a person might d£ while attempting to solve 
one of the problems. For example, "My first approach was to list all 
the facts given about each individual or role." (Ref lectlon-impu Isl vl ty. ) 
The anxiety-state items followed a similar format. The construction of 
ths questionnaires covered several steps. First, a number of items were 
written consistent with the theoretical constructs. Those statements 
judged most representative of the style dtmsnslons and most likely to 
be descriptive of the thinking processes of college students were In- 
cluded on preliminary questionnaires. As a result of a pilot study, 
some items were deleted and others revised for the final scales. 

The same theoretical dimensions were represented on the questionnaire 
prepared for each type of problem. The scales were designed to measure 
the different processes the subjects used while working on the problems, 
and the degree to which they used some processes In preference to others. 
Both questionnaires employed a true-false format, and about half the Items 
were written In a positive direction, about half In a negative direction* 
The Verbal Problem Questionnaire (VPQ) contained 45 items, and the Numeric 
Problem Questionnaire (NPQ) contained 34 items. The two scales he' 27 
items in common that contained Identical or nearly Identical wording. The 
VPQ contained 18 Items not represented on the NPQ and the NPQ contained 
7 Items not found on the VPQ. The differences In Item content of the 
questionnaires reflects content differences between each problem type. 
The items on both questionnaires were grouped according to the dimensions 
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they were judged to represent. The number of items on each quest lonna fro 
judged to represent each of the style and anxiety dimensions is given in 
Table 6. 

Procedure . The subjects were 114 male undergraduates enrolled In 
introductory psychology courses at Michigan State University. The sub- 
jects were randomly assigned to two groups of 57 subjects, and were 
tested in two sessions spaced one day apart. The design used was a com- 
pletely counterbalanced 2x2 repeated measures Latin Square with two levels 
of stress (stress vs. non-stress). Cne group was given instructions de- 
signed to induce stress (Stress Condition - told they were taking an iQ 
test, and given some unsolvable practice problems) in Session I and 
instructions designed to reduce stress (Non-Stress O^ndition - told they 
were working on some problems that would allow the Investigator to study 
their thinking styles, and given solvable practice problems) In Session 
II. The second group of subjects received the treatments in opposite order. 

Two "parallel" forms of the experimental materials were prepared, 
and subjects received Form A in Session I and Form B in Session II. Each 
form consisted of a set of premeasures in the form of personality and 
"abiiity" test measures of the style dimensions, one of each of the two 
problem types (e.g., V|N() and strategy assessment questionnaires, and 
measures of trait, test, and State Anxiety. 10 The subjects filled out 
each strategy assessment questi onnai re Immediately after finishing work 
on each prob I em. 

Data analysis . The stress manipulations had no reliable or signi- 
ficant effects on problem-solving performance (Thomander, 1972). For 
the purposes of the analyses presented in this study, the daf' wai> pooled 
across groups on each session. The measures may thus be considered as 
obtained from one group of 114 subjects on two different occasions. The 
problem-solving questionnaires were subjected to a detailed psychometric 
analysis using Hunter and Cohen's (1969) PACKAGE system of computer rou- 
tines for the analysis of correlational data. The first step In the 
analysis consisted of on oblique multiple groups factor analysis on 
the hypothesized style and anxiety Item groupings. These a priori clus- 
ters failed to meet the criteria for unique homogeneous factors. The 
Items groupings were then revised and factor analyzed several times un- 
ti I homogeneous clusters appeared. The following criteria were used in 
identifying unique homogeneous clusters: (a) Internal conslstency--a I I 
the items In a cluster must be correlated more highly with their own 
cluster than with any other clustt>r, and coefficient alpha, which Indi- 
cates homogeneity within the cluster (Cronbach, 1951), should be reason- 
ably large, (b) External parallelism — the sign and magnitude of the 
correlation of all Items within a cluster with other clusters should be 
approximately the same, (c) Homogeneity of cluster content — It should 
be logically reasonable that the items within a cluster share some common 
varl ance. 



'^Data obtained from all of these instruments will not be dis- 
cussed In this report. A complete description of all of the measuring 
instruments can be found In Thomander (1972). 
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Results and Discussion 



Can the processes the subjects reported using as they worked on the 
problem-solving tasks be mosf accurately depicted as representing style 
or cognitive strategy variables? This question is revelant to the value 
or utility of conceptualizing certain cognitive processing variables as 
"styles." If evidence for stylistic variables Is not found In problem- 
solving behavior, then the general applicability of the notion of "styles 
of processing" Is necessarily reduced. Two kinds of evidence relevant 
to this Issue were examined: (a) The structure obtained from the clus- 
ter analyses of the subjects* responses to the strategy assessment ques- 
tionnaires — did the hypothesized cognitive style dimensions appear, or 
did more specific strategy dimensions summarize the subjects' problem- 
solving processes? (b) The generality and consistency across problems 
of the cognitive processes used by the subjects in their, attempts to 
solve the problems — to what extent did the subjects respond to parallel 
tasks (the Verbal or Numeric problems) and different tasks (Verbal vs. 
Nunjeric problems) with the same or similar processes? The first line of 
Inquiry Is concerned with whether the structural analysis of the subjects 
self-reports of the processes they used in problem solving provide any 
evidonce for the theoretical style dimensions. The second line of evi- 
dence Is concerned with whether the subjects' cognitive processes are 
employed In a manner consistent with the assumption that cognitive styles 
represent general Izable and consistently used modes of processing In- 
fonnatton . 

Styles or strategies: the structure of the strategy assessment 
questlonnal res> A Separate cluster analysis was performed on the strategy 
assessment questionnaire data from all subjects In Session I, and then 
again using the data from ail subjects In Session II for each of the four 
problems. (V|, V2, and N|, N2). This procedure allowed a comparison of 
the cluster structures within each kind of problem (Numeric and Verbal) 
and between each kind of problem (Numeric vs. Verbal). The obtained clus 
ter structure was virtually identical for each set of data from the VPQ 
and for each set of data from the NPQ.'' The structures of the VPQ and 
NPQ were also found to be similar, with seven common clusters found be- 
tween them. 

Twelve clusters were obtained from the VPQ and the NPQ. Seven of 
the clusters were represented on both questionnaires, and five were 
present only on the VPQ. These five clusters were, for the most part, 
made up of Items appearing only on that questionnaire. 

The Items from both questionnaires which make up the twelve ob- 
tained clusters are shown In abbreviated form In Table 5. The words 
judged to contain the essence of each Item have been listed In the genera 
order of their loadings on each cluster. Below is a brief description of 
the dimension each cluster Is thought to measure as judged from Item con- 
tent. The names of the cluster were chosen to summarize the dimension 
measured by each cluster. Clusters that contained the same or very simi- 
lar items on both the VPQ and NPQ were given the same name. 



''See Thomander, Hunter, & Marshall (1975) for a complete presen- 
tation cf the C'Juster structure and cluster tables. 
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TABLE 5 

STRATEGY ASSESSMENT QUESTIONNAIRE CLUSTERS 



). Frustrated ; Felt frustrated, discouraged, did aot enjoy working 
on the problem, was not easy to solve, worried might not solve it, 
wondered how well others were doing, problem could have more than oae 
right answer. (V,N} 

2. Concentrated : Did not have trouble concentrating, were no tfmes when 
cou Idn't seem to think, mind did not wander, did not quit working on 
the problem, thought would be able to solve It, did not guess before 
time was up. (V,N) 

3. Enjoyed : Enjoyed working on the problem, took It as an interesting 
cha I lenge. (V) 

4. Strategy Change ; Changed strategy or approach at least once, changed 
more than once, worked problem more than once, didn*t expect to solve 
by logical deduction, problem not easy, overlooked some Important ele- 
ment, got more than one answer, changed answer. (V,N) 

5. Caref u I : Did not guess before time was up, did not change answer, 
did not stick with an unproductive hypothesis a long time, did not 
write down answers until positive alt facts checked out, careful 
and cautious, did not check out improbable hunches. (V) 

5. De 1 1 berate : Read problem several times before deciding how to atl.jcu 
ft, tried to visualize overall picture, thought about briefly before 
began calculations, suspected a trick solution, did not forget about 
the persons and objects mentioned and figure with numbers oniy. (V,N) 

7. Globa I ; Did not concentrate on details, read problem over and over 
in effort to get the entire picture, concentrated on generalizations 
or assumptions could make from facts^ suspected trick solution, tried 
to visualize entire problem as a unit. (V) 

8. Notes: Used pencil and paper to make notes as worked on problem, 
wrote most calculations down on paper, did not work most calculations 
in head. (V,N) 

9. Systematic : Used a rational systematic approach at all times, had 

a specific plan in mind, never abandoned logic to use a nonsystematic 
approach, tried to use a rational systematic plan. (V,N) 

10. Trial and Error : First approach did not Involve listing all the facts, 
first strategy was basically trial and error. (V) 



II. New Approach : First approach 
on particular demands of this 
this specific problem. (V,N) 



not based on previous experience, based 
problem, seemed appropriate to use for 



12. Fixated ; Used same hypothesis again and again, did not quickly aban- 
don a hypothesis when found It ran counter to the facts, (V) 



Note. V= cluster on VPQ 
N=^ cluster on NPQ 
O 39 

ERIC 



The letters at the end of each description indicate which question- 
naire the cluster was obtained from. 



Frustrated . Indicator of discouragement over a problem with which 
the subject experienced some difficulty and worry about how well he was 
doing in comparison to others. {V,N) 

Concentrat ed . Indicates whether or not the subject was having diffi- 
culty keeping his attention on the problem-^solving task. (V^N) 

Enjoyed . Measures how much the subject liked working on the prob- 
lem, taking it as an interesting challenge. (V) 

Strategy Change. Is a measure of whether the subject tried more 
than one strategy or approach for solving the problem. The content of 
the cluster suggests that it measures strategy changes associated with 
the subject having difficulty solving the problem. (V,N) 

Caref u I . Indicates an approach exemplified by cautiousness, lack 
of guessing, and avoidance of unlikely hypotheses. (V) 

Del 1 be rate . Assesses the extent the subject took time to think 
about how to begin, what Is usually meant by a reflective approach. 
That is, to take careful thought as how to attack the problem Instead 
of Impulsively beginning with the first Idea that comes to mind. 

Globa I . Measures whether the subject attempted to conceptualize the 
entire problem as a unit, or whether he concentrated on details. It was 
found only on the VPQ, and even there It was one of the weakest clusters. 
The two Items on the NPQ written for the global-analytic dimension be- 
came part of the Deliberate cluster on that questionnaire. Some of the 
time taken before beginning the Numeric Problems (which Is measured by 
the NPQ Deliberate cluster) may have been used by some subjects to try 
to get an overall picture of the problems before deciding on an approach. 
On the other hand, the different clustering on the two questionnaires may 
be due more to differences In the number of Items written for the global - 
analytic dimension (2 on the NPQ, 7 on the VPQ) than to differences In 
the two types of prob letvt-solvl ng tasks. (V) 

Notes . The meaning of the Notes cluster Is straight forward— el ther 
the subject made notes while working on the problem or he did ail his 
thinking without the aid of pencil and paper. (V,N) 

SystematI c . Assesses the use of a rational or logical and systema- 
tic plan for solving the problem. (V,N) 

Trial and Error . Concerns the use of a non-systematic or trial and 
error approach. (V) 

New Approach^ Indicates whether the first approach used on the prob- 
lem was chosen to fit the particular demands of that problem or whether 
the first approach was one. the subject i^abltually uses on this kind of 
task. (V,N) 

FJxated. Measures perseveratl ve behavior in problem solving; that 
is, using tWe same hypothesis over and over again even when It doesn't 
seem to be leading to an appropriate solution. (V) 

It Is clear from the analysis of cluster content that the clusters 
represent processing dimensions that are different from the hypothesized 
style dimensions, and the anxiety states. Instead of finding four unique 
dimensions among the questionnaire items, twelve were obtained. The di- 
mensions obtained are. In at least some cases, more narrow or circum- 
scribed than the dimensions anticipated. Table 6 shows what occurred, 
and gives the distribution of questionnaire items among the obtained and 
hypothesized clusters. Across the top of the Table are .the four a priori 




40 



TABLE 6 

DISTRIBUTION AND COMPARISON OF PROBLEM-SOLVING QUESTIONNAIRE 
ITEMS FOR HYPOTHESIZED AND OBTAINED CLUSTERS 



Number of Items tn A Priori Item Groupings 





1- lexi 


Kilt +v/ 

Dl LLTy 


rseT 1 ecT 


1 veness 


Globalness 


Anxiety 


NPQ 


VPQ 


Npg 


VPQ 


NPQ 


VPQ 


NPQ 


VPQ 


Processes; 


















New Approach 


2 


2 














Strategy Change 


5 


2 


1 


1 




1 


1 




Trial & Error 




2 














Fixated 




2 
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Del (berate 
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2 




1 




Careful 




1 




5 










Global 












A 




1 


States : 


















Frustrated 


1 












5 


5 


Concentrated 






1 








5 


4 


Enjoyed 
















2 


Residual 


3 


3 


1 




2 


1 


1 


Total 


6 


12 


1 1 


13 


2 


7 


13 


13 
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dimensions the items were written to measure. Across the bottom of the 
table are the total number of items representing each a priori dimension. 
Most of the a priori flexibility-rigidity items formed four clusters — 
New Approach, Strategy Change, Trial 4 Error, and Fixated. Most of the 
a priori ref lection-lmpu Isivlty items formed four clusters — Notes, 
Systematic, Deliberate, and Careful. Most of the a priori global-analytic 
Items from the VPQ formed the Global cluster on that questionnaire, and 
the a priori global-analytic Items from the NPQ joined the Deliberate 
cluster on that questionnaire. The majority of the a priori anxiety 
state items formed three clusters — Frustrated, Concentrated, and Enjoyed. 

For the most part, the clusters that emerged from the data appear 
to be a refinement of the partition determined by the a priori dimensions, 
suggesting that the clusters in each group derived from the items of 
a single a priori dimension may represent different manifestations or 
aspects of that dimension. Certainly the content of the clusters suggests 
that they represent different aspects of the theoretical dimensions. For 
example, "Taking notes," "Being systematic," "Deliberate" and "Careful" 
all appear to be' consistent with the functioning of a reflective style; 
and "New Approach," "Strategy Change," "Trial and Error," and "Fixated" 
all appear to represent different properties of Flexibility-Rigidity. 
If the clusters that were obtained from items written to measure the 
same dimension reflect different aspects of that dimension, then these 
clusters should share some common variance, and they should all be posi- 
tively corre{ated» That Is, the original hypothesi 2i:ed dimensions should 
appear as "second-order factors," when the tntercorrelations of the em- 
pirically obtained clusters are examined. 

Table 8 presents the correlations between the summed two-problem 
cluster scores (V1+V2; N1+N2) grouped according to the a priori dimensions 
from which the items were derived. The group of anxiety clusters show 
the pattern of consistent significant relationships which Indicates that 
all the clusters In the group are related to the same dimension. These 
three clusters were also found to be highly related to the other anxiety 
measures used In the larger study (Thomander, 1972). Neither the flex- 
ibility nor the global group of clusters show this pattern. The reflec- 
tiveness group does have some significant intercorrelations, but the 
pattern, and direction of the relationships is not consistent across both 
the VPQ and NPQ. Thus, there 5s no strong case for a common dimension 
among all the clusters In the reflectiveness group. 

It is very unlikely that the low intercorrelations found among each 
group of clusters derived from the same a priori style dimension are due 
to measurement error. As is shown in the next section, the random error 
in response to the questionnaires averaged only 27 percent of the variance 
In the summed two-problem cluster scores. Correlations between the two- 
problem cluster scores presented In Table 7 could^ therefore, have been 
as high as ,73. 

In summary, the clusters were not found to be correlated as would be 
predicted on the basis of their relation to the a priori style dimensions. 
One especially Interesting set of four clusters was found that were quite 
consistently I ntercorre lated both within and between questionnaires. They 
are presented In Table 8. 

. Although the clusters presented In Table 8 are significantly related, 
the direction of the correlations and their combined content does not 
suggest a single cognitive style dimension. The directions of the corre- 
lations show that frustration and changing stratecjies are associated with 
poor concentration and lack of carefulness. One possible Interpretation 
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TABLE 7 

CORRELATIONS BETWEEN CLUSTERS WITHIN EACH OF THE 
A PRIORI COGNITIVE STYLE AND ANXIETY GROUPS 



Fiexibil ity-Rlgldity VNA VSC VTE VF VNE NSC 

Verbal New Approach 100 



Verbal Strategy Change 


04 


100 








Verbal Trial and Error 


01 


06 


100 






Verbal Fixated 


-06 


12 


-14 


100 




Numeric New Approach 


28»» 


02 


08 


-.08 


100 


Numeric Strategy Change 


10 


2j_» 


-16 


-10 


09 



Ref lection-lmpulsivity 
Verba I Notes 
Verbal Systematic 
Verbal Deliberate 
Verbal Careful 
Numeric Notes 
Numeric Systematic 
Numeric Deliberate 



VN 



VS 



VD 



VC 



NN 



NS 



NO 



100 

II 100 

-09 -22» 100 

-14 46*** -09 

32*** 08 -04 

-08 - 1 2 

-07 05 07 



100 

04 100 
36*** 17 

00 21* 



100 
19* 



100 



Global -Ana lytic 
Verbal Global 
Verba I Del i berate 
Numeric Deliberate 



VG 



100 
13 
18 



VD 



100 
07 



ND 



100 
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Anxiety States VF VC VE NF NC 

Verbal Frustrated 100 

Verbal Concentrated -3|»»» |00 

Verbal Enjoyed -20* 27** 100 
Numeric Frustrated 50*** -22* -25** 100 

Numeric Concentrated -28** 3|«»» 23* -57*** 100 

Note. Cross correlations are underlined. 
* p < .05 
** p < .01 
p < .001 



TABLE 8 



CONSISTENTLY INTERRELATED CLUSTERS 
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Frustrated 
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Concentrated 


2N 


-28** 
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34»»» 
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Strat. Chg. 


4N 


25** 


-04 


2I_* 


-05 


38*** -28* 


* 100 

















Note. Cluster 5V was not represented on the Numeric Problems Question- 
naire. 



*p < .05 
< .01 
***p < .001 



Is that subjects wtth high anxiety do not control their attention span, 
become hurried and change strategies often, and hence feel frustrated. 
It is equally possible, of course, that when a subject begins to feel frus- 
trated, worry, etc., his attention to the task is Interrupted, and he be- 
gins to change strategies and hurry through the problem. Wine (1971) 
has proposed a cognitive view of the effects of anxiety on performance, 
which agrees with the latter interpretation. She argues that a high test- 
anxious person Is internally focused on sel f-eva luative worry, frustration, 
etc., -'^nd hence, that his attention is directed away from task relevant 
variables and his performance Is thereby lowered. 

In summary, no evidence was found in the cluster analyses of subjects* 
reports of the processes they usedjto indicate that broadly defined sty- 
listic variables played a role in their problem-solving behavior. The re- 
sult was unexpected since many of the questionnaire items were written to 
tap relatively high level processes that could cover a wide range of cog- 
nitive behavior. Items were written to assess three cognitive style 
dimensions, and ft was anticipated that the dimensions underlying responses 
to the questionnaires would closely parallel the a priori dimensions. In- 
stead the cluster analyses resulted In the Isolation of a number of pro- 
cessing dimensions, each of which concerns an aspect of cognitive behavior 
that is more specific than a cognitive style. Moreover, the size of the 
correlations obtained between the clusters in each style grouping were 
found to be more indicative of independent strategy-like dimensions than 
different aspects of the same factor. Only the anxiety clusters were 
found to correlate with each other In a way suggesting that they share a 
common dimension. 
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styles or strategies; consistency in the use of jprocesses on 
different problems ^ No evidence of the first kind sought for the ex- 
pression of cognitive styles in the problem-solving behavior of the sub- 
jects was obtained. We now turn to an examination of the second kind 
of evidence needed to decide whether or not an identified processing 
dimension is representative of stylistic variables, namely the manner 
in which the processes represented by the clusters were used by the sub- 
jects. The analysis comprised three steps. The first two steps Involved 
breaking dC)Wn the variance in questionnaire cluster responses to its 
sources, and determining the degree of genera I i zabi I ity in responding in 
each cluster dimension across ,prob I ems of the same type and different 
types. First, sources of variance for the cluster scores on single prob- 
lems were determined and the degree strategy usage generalized across 
problems was calculated* The second step involved performing the "same 
kind of analysis, although in less detail, for each pair of problems of 
the Same type (V1+V2 and N1+N2) to maximize the possibi I ity that sty- 
listic consistencies in processing might be found. If the dimensions 
measured by the clusters have a strong stylistic component, then the 
variance breakdown should show that their usage is linked more closely 
to the subjects' tendency to use the same processes consistently across 
time and different problems than to the unique characteristics of in- 
dividual problems or types of problems. 

In the third step, changes in cluster score means over the 24 hour 
period between testing sessions are discussed. If the dimensions mea- 
sured by the clusters represent stylistic variables, then their use 
should be stable over time. If, on the other hand, the clusters repre- 
sent strategies, then their usage should change as a result of experience 
in working on the problems in session I. That Is, stylistic processes 
should be used consistently and be relatively unaffected by a single 
testing experience; but strategies, which are chosen on the basis of 
task characteristics and past learning experiences, would be expected to 
show practice effects under the same circumstances. 

Two estimates of cluster reliability were obtained, coefficient 
alpha (which gives an estimate of reliability for each cluster for each 
problem) and the parallel form reliability (r\ 2^* Table 9 represents 
the reliability data for the various clusters.' Since coefficient alpha 
for an average cluster over both types of problems is .65 (.61 + .71 / 2 = 
.65), random error (I - .65), on the average accounts for about 35^ of 
the variance in the subjects' questionnaire responses for each problem, 
while 65^ of the variance (coeff. alpha = .65) Is reliable variance. 12 
The amount of error variance changes from cluster to cluster, being high- 
est (55?) for the five item NPQ "Deliberate" cluster and lowest (16?) for 
the two Item VPQ "Systematic" cluster. Because the questions were de- 
liberately pitched at an abstract level, for eximple, "were y-t^Q sys- 



It Is generally known that the reliability coefficient Is the per- 
cent of total variance in a measure that Is true score variance (e.g., see 
Nunnally, 1967). (Hence, 65? of the average cluster variance for a single 
problem is reliable Ctrue score] variance, and random error is I -.65 or 
35?.) This fact is used in several places in this section to break down 
the total variance in subjects' questionnaire responses (for the clusters) 
into variability that is random error, variability that generalizes across 
problems and variability that is specific to particular problems. 
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TABLE 9 

STRATEGY ASSESSMENT QUESTIONNAIRE CLUSTER RELIABILITIES 



Cluster 


Coef , 


'Alpha 




Session 
1 


1 Session 
2 





Vert3a I rroD I ems 








r rusTraTea 


AQ 






Concentrated 


.55 


.57 


• 27 


Enjoyed 


• 66 


• 79 


.27 


Strategy Change 


.54 


.58 


.31 


Carefu 1 


.59 


.61 


.04 


De 1 i berate 


.66 




.J 1 


Globa 1 


.42 


.54 


.41 


Notes 


.52 


.68 


.22 


Systemat 1 c 






.15 


Trial and Error 


.43 


.53 


. 13 


New Atpp roach 


.50 


.61 


.16 


Fixated 


.48 


.48 


.31 


Numeric Problems 








Frustrated 


.74 


.69 


.19 


Concentrated 


.71 


.64 


.31 


Strategy Change 


.79 


.84 


.03 


Del I berate 


.36 


.54 


.22 


Notes 


.83 


.85 


.19 


Systematic 


.73 


.61 


.31 


New Approach 


.70 


.72 


.36 


Average 


.61 


.70 


.23 



ERIC 
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tematlc"? rather than at a concrete level, such as, "dfd you ever con- 
sider the daughter for the role of murderer"?, the subjects may have 
had some difficulty in translating their experience into verbal reports- 
In general, however, the coefficient alphas Indicate reasonably good 
reliabilities for the single problem clusters. 

Table 9 also provides evidence of the degree of consistency in the 
subjects' use of specific cognitive processes (as described by the clus- 
ters) across problems of the same type, that Is, on the Verbal and on 
the Numeric Problems. R|2, the parallel form reliability, (the question- 
naires for each problem type were identical, and the problems of each 
were designed to be parallel forms of approximately equal difficulty; 
thus the correlation R|2 can be considered a type of parallel form re- 
liability based on the actual and apparent comparability of the measur^ 
ing devices.) is the correlation between the cluster scores for the two 
problems of each type, and indicates the extent the subjects reported 
using the same cognitive processes on each of the two Verbal Problems 
and on each of the two Numeric Problems. The average parallel form re- 
liability is only. 23, suggesting that there was considerable shifting 
in the subjects' mode of attack on the problems. This number, however, 
is of sufficient magnitude to suggest that there is a degree of generality 
in the subjects' us© of processes across problems of the same type. 

On the other hand, when a comparison is made between the average 
parallel form reliability (.23) and average coefficient alpha (.65) in 
terms of variances (see footnote 12), it can be seen that only 25% of 
the total variance (or .23/. 65, i.e., 35^ of the reliable variance) 
generalize across problems of the same type, while 42% (.65-. 23) of 
the total variance (or I-.35, i.e., 65^ of the reliable variance) is 
specific to particular problems and represents the degree of cluster in- 
stability across problems of the same type. Thus, the subjects' re- 
ported processes for any one problem are in targe part specific to that 
particular problem, rather than indicative of a general tendency to re- 
spond consistently with the same (or a similar process) to all problems 
of that type. On the average, only about 25% of the total variance, 
or 35^ of the reliable variance, in the subjects' questionnaire responses 
to any one problem is accounted for by a tendency to report responding 
to both Verbal or both Numeric Problems with the same process. 

Since some consistency in the subjects' reported processing was found 
for problems of the same type, it can be asked whether or not the sub- 
jects' reported consistency in the use of processes on Verbal Problems 
is the same as their rtjported consistency on Numeric Problems. That Is, 
to what extent does the variance in the subjects' questionnaire responses 
that generalizes across problems of the same type also generalize across 
problems of different types? The relevant cross correlations between 
single problems of different types are presented in Table 10. Note that 
because there are two Verbal and two Numeric Problems, there are four sets 
of cross correlations between them. The average correlation across prob- 
lems of different types is .17, which is almost as large as the average 
correlation between problems of the same type, that is, .23, suggesting 
that most aspects of the subjects' cognitive processing that generalize 
across problems of the same type also generalize across problems of 
different types. 

The average correlations between cluster scores for problems of 
different types <rvn) is compared to the correlations between problems 
of the same type (rv|V2, ^r\\^2^ ''^ Table II. Of particular interest Is 
the last column, which givei the average correlation between the corres- 
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TABLE 10 

CROSS CORRELATIONS BETWEEN CORRESPONDING CLUSTER 
SCORES FOR VERBAL AND NUMERIC PROBLEMS 



Cluster^ 


V,.N, 


V,,N2 


V2'N| 


V2,N2 


Average 


Frustrated 


.27 


.34 


. 14 


.45 


.30 


Concentrated 


. 16 


.27 


.08 


.28 


.20 


Strategy Change 


.09 


. 04 


. 12 


.23 


. 1 2 


Del i berate 


-.06 


,00 


.01 


.23 


.05 


Notes 


.24 


.20 


.21 


.13 


.20 


Systematf c 


.04 


. 17 


.05 


.25 


. 13 


New Approach 


.05 


.34 , 


.03 


.25 


. 17 


Average 


. 1 1 


. 19 


.09 


.26 


.17 



Note. The notations V| and V2 stand for the first and second verbal 
problems. N| and N2 are ihe two numeric problems, 

aOnly the clusters that were represented on both the NPQ and VPQ are 
presented. 



TABLE I I 

COMPARISON OF THE CROSS CORRELATIONS BETWEEN 
CORRESPONDING VERBAL AND NUMERIC CLUSTER SCORES WITH 
THE CORRELATIONS BETWEEN CLUSTER SCORES FOR PROBLEMS OF THE SAME TYPE 



Cluster rv,v2 rN|N2 ^^vn^® ryj^J 

Frustrated .22 .19 .30 1.47^ 

Concentrated .27 .31 .20 .69 

Strategy Change .31 .03 .12 1.26^ 

Deliberate .31 .22 ,05 .19 

Noras .22 .19 .20 1.00 

Systematic .15 .19 .13 .76 

New Approach ,16 .36 .17 .71 



Average .23 .21 .17 .87 



a. The estimated true score correlation between clusters for different 
problem types is given by the ratio: 

^VooNc. = ^Vi/Y ^V|V2 ^N,N2' 
to within sampling error. 

The estimated values greater than one were rjiiot set to 1.00^ in order 
to provide the maximum possible estimate of the average relationship 
between VAN, . 




ponding Verbal Problem and Numeric Problem cluster scores corrected for 
attenuation (ryjvj^). The corrected correlation gives an estimate of the 
degree of consistency in the subjects' reported use of cognitive processes 
across problems of different types. For three of the clusters (Frustrated, 
Strategy Change, and Notes), the correction for attenuation produces an 
estimated correlation of 1.00. Thus, for these three clusters there Is 
no distinction between the consistency with which subjects' reported re- 
sponding to Verbal and Numeric Problems. Only for Deliberate is there 
a sharp cleavage between the subjects' reported use of strategies on the 
two types of problems. On the average, Bl% of the reliable variance in 
subjects' questionnaire responses that is due to consistencies in their 
reports of processes used on problems of the same type also generalizes 
across problems of different types. 

A summary breakdown of the total variance in the subjects' responses 
to an average cluster for one problem can now be given. Since the av.erage 
reliability of a cluster .for a single problem is .65 (coeff. alpha), the 
average amount of the variance due to random error is 35^ (I -.65). Of 
the 65^ of the variance that 5s reliable variance, one third (35?), or 
21% of the total variance, represents consistencies in responding to all 
problems of a given type; and 65% (I -.35) of the reliable variance, or 
42? (.65-. 23) of the total variance, represents responding to the unique 
aspects of each individual problem. Of the total variance which is shared 
by problems of the same type (23?), fully 87? is common to problems of 
the opposite type. That is, 20? (.87 x .23 = .20) of the total variance 
Is subjects' responses to the average cluster for one problem Is accoun-* 
ted for by an individual difference variable that assesses the subjects' 
tendency to react consistently to all problems in the manner indicated 
by that cluster. In other words, 20? of the total variance represents 
the degree all four problem-solving tasks were parallel in terms of the 
cognitive processes subjects reported using while working on them. It Is 
not surprising that some parallelism was found, since all of the problems 
were deductive reasoning tasks of approximate equal difficulty. It 
should be emphasized, however, that It Is not the parallelism of the 
tasks that Is In question, but the general i zabl 1 1 ty of the procesises used 
on those tasks. Although It Is possible that the 20? of the total vari^ 
ance that is held in common with ali the tasks could be due to actual 
similarities in the problems, this Is the only portion of variance that 
can be accounted for by individual consistencies in the use of processes 
that generalize across different problems. Hence, it is to this portion 
of the variance that we look for evidence of the expression of cognitive 
styles. 

In surmiary, the data for single problems indicate that the greater 
portion of the subjects' reports of processing are a function of the 
unique characteristics of individual problems; but the evidence also 
solidly indicates that each cluster ^core assesses. In part, an aspect 
of the subjects' problem-solving processes that are consistent across 
problems, across problem type, and across time. 

Sources of variance In tw o-problem cluster scores. In order to 
determine whether or not the generality In reported processing could be 
taken as evidence for the expression of cognitive styles by the subjects, 
a more reliable Index of the subjects' mode of responc^^ng to the problem- 
solving tasks were sought. Therefore, the subjects' cluster scores over 
the two problems of the same typo were summed (V1+V2 and N|+N2>. The 
maximum possible correlation batween two such scores is determined by the 
root mean square of their "reliabilities". The various "reliabilities" 
are presented in Table 12^ The reliabilities are set up to break down 
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TABLE 12 

RELIABILITY COEFFICIENTS FOR CLUSTER SCORES 
SUMMED ACROSS TWO PROBLEMS OF THE SAME TYPE 



Cluster 


Alpha^ 


r b 
V V 

^r2 


^n,n/ 


^vn'' 


Frustrated 


.72 


.36 


.32 


.50 


Concentrated 


.69 


.43 


.47 


.31 


Strategy Change 


.74 


.47 


.06 


.21 


Del i berate 


.65 


.47 


.36 


.07 


Notes 


.78 


.36 


.32 


.32 


Systematic 


.81 


.26 


.47 


.21 


New Approach 


.71 


.28 


.53 


.28 



Average .73 .38 .36 .27 



^The amount of nonrandom variance in each two-problem sum was calcula-* 
ted by using the formula fc'^ the reliability of the sum of two corre- 
lated measures. 

'^The correlation between the cluster scores for. the Verbal Problem on 
each session stepped up by the Spearman-Brown split half formula. 

^The correlation between the cluster scores for the Numeric Problem on 
each session stepped up by the Spearman-Brown split half formula. 
The correlation between the sum of the cluster scores for the two Verbal 
Problems and the sum of the cluster scores for the two Numeric Problems. 



the variance in a two-problem sum. Also presented are the correlations be- 
tween cluster scores for the two problems of the same type (stepped up by 
the Spearman-Brown split half formula) and the correlation between the sum 
of the cluster scores for the two Verbal Problems and the sum of the cluster 
scores for the two Numeric Problems. Each entry in the column Icbeled 
"Alpha" is the percent of variance left after random error Is removed. 
This would be the appropriate "reliability" for the correlation between the 
two-problem sum and a variable that is correlated with the problem specific 
component of the subjects* reports. The entries In the columns labeled 
"'"VIV2" """nino" represent the amount of variance accounted for by the 
subjects' tendency to react in the same way to all problems of the same type. 
These are the appropriate "reliabilities" for variables that can be assumed 
to be uncorrelated with the problem specific component of the subjects V 
reaction. Finally, the entries In the column headed 'Vyn*" which are the 
cross correlations between the two-problem sum for the Verbal and Numer;Ic 
clusters, can be considered the appropriate "reliabilities" for variables 
that are correlated only with the subjects* tendency to react^in the same 
way to a I I p rob I ems. 
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A variance breakdown for the typical two-problem cluster score can 
now ivB given, and is summarized in Table 13. The average Alpha is .73, 
so the average amount of the total variance due to random error In the 
subjects' self-reports is 21% (I-.73). The average amount of the total 
variance that is due to a tendency for subjects to report reacting in the 
same way to all problems of a given type is .38 + ,36 / 2 = 375^. And 15% 
of that (.27/. 37 = .73) variance (i.e., 27^) is also common to problems 
of the opposite type. Or putting ir the other way around, 10^ (.37 - .27) 
of the iota I variance is due to responding that is consistent across prob- 
lems, but specific to verbal or numeric content. Finally, 36^ of the 
total variance (.73 - .37 = .36) is accounted for by that part of the sub- 
jects' reactionsthat are specific to the unique characteristics of each 
individual problem. 

TABLE 13 

TOTAL VARIANCE BREAKDOWN FOR SINGLE PROBLEM 
AND TWO-PROBLEM CLUSTER SCORES 



Source of Variance Single Problem Scores Two-Problem Scores 



Random error 


.35 


.27 


Specific to each individual 
prob lem 


.42 


.36 


Genera 1 to a 1 1 prob 1 ems^ 


.20 


.27 


General, but specific to^- 
problems of the same type 


.03 


.lO'' 


Total 


1 .00 


1 .00 



a. Note that: .20f.03=23^ & .27+.IO=37^, the amount of total variance 
that generalizes across problems of the same type. 

b. For three clusters (Frustrated, Strategy Change, Notes) there Is no 
content specific factor. Thus the 105^ is an average based largely 
on the 2A% for Deliberate. 



As shown in Table 13, the effect of using the two-problem cluster score, 
rather than single problem cluster scores, in computing a variance break- 
down has been to increase from 20 to 275^ the portion of total variance 
that can be accounted for by tendencies of the subjects to report using 
the same processes on all problems. As with the single problem cluster 
scores, in the subjects' reports of the processes they used, there is 
still a larger portion of the total variance attributable to specific 
problem factors than to consistencies that generalize across all problems. 
It is concluded, then, that evidence of the second kind sought for the 
expression of stylistic variables in problem-solving behavior was not 
found. On the contrary, the subjects' reports of problem-solving pro- 
cesses are more dependent on tasks than subject parameters, suggesting 
that the variables measured by the clusters behave like strategies rather 




The validity of the measures of cognitive processes . What evidence 
exists, other than already presented, that the "factor-analysis" of self- 
reports obtained by a systematically constructed questionnaire actually 
reflect aspects of the subjects' cognitive processing. Good evidence 
already exists that such measures provide reliable and valid evidence 
of cognitive functioning (Frederiksen, 1969, 1970; Marshall, 1971). 

In thfs study, several points can be made that support the conclu- 
sion that the cluster dimensions validly tap aspects of cognitive func- 
tioning. First, in reference to the cluster Intercorre latlons, it is 
very unlikely that the low i ntercorre latlons found among each group of 
clusters derived from the same a priori style dimension represent a fail- 
ure of measurement. The average error in response to the questionnaires 
averaged 21% of the total variance in the two-problem cluster score. 
Correlations between single clusters could, therefore, have reached .73 
(I-.27). Furthermore, the variance that did generalize across time also 
generalized across different types of problems. Thus, the observed re- 
sponse consistencies appear to reflect the subjects' general problem- 
solving behavior on tasks of this nature* 

Could It be that the post problem self-reports are In fact Indepen- 
dent of the actual problem-solving behavior and hence independent of 
cognitive strategies and styles? First, two thirds of the reliable vari- 
ance in responses was specific to the particular problem. Therefore, the 
subjects could not have been responding solely to the questionnaire. 
Second, mean differences were found in subject responding to the clusters 
of questionnaire items as they went from Session I to Session II one day 
latar. The consistent pattern of changes across both types of problem- 
solving tasks indicate a general practice effect. It appears that the 
subjects, on the average, benefited from their experience on the Session I 
problems, and, when faced with similar problems on Session II, changed the 
Way they approached the problems to a significant degree. All of these 
findings are consistent with the typical findings of the experimental 
literature and hence tend to support the conclusion that the cluster scores 
do indeed tap the subject's problem-solving processes, while he worked 
on the problems. 

Practice effects . Each a priori dimension was represented on the 
questionnaire by items written to measure each end of the dimension. For 
example, there were items describing rigid as well as flexible approaches. 
In scoring the items for each cluster, some items were reversed so that 
all were scored In the same direction. The possible range, mean, and 
standard deviation for each cluster on each session are presented in 
Table 14. Inspection of these numbers shows that the means were at In- 
termediate values during the first session. Thus^change was possible 
In both directions for each cluster score as the subjects went from Session 
I to Session I I a day later. 

The mean changes In responding to each cluster over time are pre- 
sented In Table 15. Session I means were subtracted from Session II means. 
Therefore, if the Session II mean Is larger, the difference score has a 
positive sign and if the Session II mean Is smaller, the difference score 
has a negative sign. On Session II the subjects were, on the average, 
less frustrated and better able to concentrate. They changed strategies 
less, were less deliberate, less global, took notes more, fixated less, 
were more systematic on the Numeric Problem and used new approaches less 
on the Verbal Problem. In most cases these changes were In the same di- 
rection on both types of problem. Since it is unlikely that the problems 
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TABLE 14 

STRATEGY ASSESSMENT QUESTIONNAIRE CLUSTERS: 
MEANS AND STANDARD DEVIATIONS 

possible range Session I Session 2 

Cluster of scores Mean S.D. Mean S.D. 



Verbal Problems 



IV. 


Frustrated 


0 


- 


5 


3.07 


1 .57 


2.21 


1 .45 


2V. 


Concentrated 


0 


- 


4 


2.83 


1.13 


3.27 


1.05 


3V. 


Enjoyed 


0 


- 


2 


1.51 


0.74 


1 .47 


0.81 


4V. 


Strategy Change 


0 


- 


4 


2.25 


1.18 


1.77 


1 .23 


5V, 


Caref u 1 


0 


- 


6 


3.40 


1 .62 


3.77 


1.58 


6V. 


De 1 i berate 


0 


- 


2 


1.13 


0.85 


0.89 


0.88 


7V. 


Global 


0 


- 


5 


2.79 


1 .22 


2. 13 


1,39 


8V. 


Notes 


0 


- 


2 


0.80 


0.83 


1 .35 


0.81 


9V. 


Systematic 


C 




2 


1 .06 


0.93 


I.I 1 


0.92 


lOV. 


Trial and Error 


0 




2 


1 .00 


0.78 


0.89 


0.80 


IIV. 


New Approach 


0 




2 


L36 


ore 


0.86 


0.83 


12V. 


F1 xated 


0 




2 


1 .07 


0 9 


0.75 


0.76 


Numeric Problems 
















IN. 


Frustrated 


0 




6 


2.08 


1 .86 


0.67 


1. 16 


2N. 


Concentrated 


0 




6 


5.05 


1.32 


5.67 


0.82 


4N. 


Strategy Change 


0 




7 


3.04 


2.23 


1.53 


2.03 


6N. 


De 1 1 berate 


0 




5 


3.21 


1 .19 


1 .83 


1.37 


8N. 


Notes 


0 




2 


0.61 


0.85 


1.32 


0.88 


9N. 


Systematic 


0 




2 


1.29 


0.84 


1.66 


0.62 


1 IN. 


New Approach 


0 




2 


1.27 


0.84 


1.19 


0.87 
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TABLE 15 

STRATEGY ASSESSMENT QUESTIONNAIRE CLUSTER 
MEAN DIFFERENCE SCORES 



Cluster Name Verbal Problems Numeric Problems 
Group I Group I I Group I Group I I 



Frustrated 


-0.85*** 


-0.87*** 


-1 .38*** 


-1 .42*** 


Concentrated 


0.56** 


0.31 


0.68*** 


0.54** 


Enjoyed 


-0. 17 


0.07 






Strategy Change 


-0.57** 


-0.38* 


-1 .61*** 


-1.43*** 


Caret u 1 


0.38 


0.35 






Del 1 berate 


-0.30* 


-0.20 


-1 .24*** 


-1.53*** 


Global 


-0.63*** 


-0.68*** 






Notes 


0.52*** 


0.57*** 


0.67*** 


0.74*** 


Systematic 


0. 12 


-0.04 


0.35*** 


0.39*** 


Trial and Error 


-0. 1 1 


-0. 1 1 






New Approach 


-0.61** 


-0.37** 


0.00 


-0.16 


Fixated 


-0.35** 


-0.28* 







*p < .05 
< .01 
***p < .001 



on Session II were very different In nature from those on Session I, these 
changes probably represent general practice effects. It appears that the 
subjects, on the average, benefited fr^om their experience on the Session I 
problems, and, when faced with similar problems on Session II, changed the 
way they approached them to a significant degree. 

Thus, in this third step of the examination of processing use, the 
kinds of changes were found that would be expected If the dimensions 
measured by the clusters represent strategies rather than styles. While 
styles are, by definition, stable or consistent manifestations of cogni- 
tive processing, strategies are open to the Influence of brief encounters 
with a problem-solving task* 

Problem-solving efficiency . Now that It has been established that 
the processing dimensions measured by the clusters behave like cognitive 
stratefjles. It may be asked how they relate to measures of problem-solving 
efffcfency. For example, do subjects who proceed systematically tend to 
obtain accurate solutions? Do those who take notes tend to spend more or 
less time on each task? Do the strategies that help In solving the Verbal 
Problems also help on the Numeric Problems? Examining these relationships 
provides a task analysis of the two types of problems In terms of the 
strategies that are helpful In solving each, and gives useful Information 
about differences between the problems. 
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TABLE 17 

CORRELATIONS OF PROBLEM SOLVING QUESTIONNAIRE CLUSTERS 
WITH MEASURES OF PROBLEM SOLVING EFFtCIENCY 









1 

1 • 






4 


1 


Verbal Problems: 


Correct 


100 








2 


Verbal Problems: 


Time 


00 


100 






3 


Numeric Problems: 


Correct 


03 


-02 


100 




4 


Numeric Problems; 


Time 


16 


34* 


01 


100 




VPQ Clusters 












4 


Strategy Change 




02 


04 


-13 


03 


12 


Fixated 




-19* 


1 1 


-02 


-16 


10 


Trial and Error 




-12 


-07 


-08 


-14 


8 


Notes 




-06 




03 


01 


9 


Systematic 




07 


05 


14 


-04 


5 


Caref u 1 




21** 


-06 


25** 


09 


6 


De 1 i berate 




05 


-09 


-22* . 


-10 


1 1 


New Approach 




1 1 


-09 


-19* 


21* 


3 


Enjoyed 




06 


06 


17 


-09 


1 


Frustrated 




-06 


29** 


-06 


08 


2 


Concentrated 




23* 


07 


02 


06 


7 


Global 




03 


-07 


-06 


-09 




NPQ Clusters 














Frustrated 




-07 


-04 


-02 


20* 




Concentrated 




08 


04 


12 


-02 




Strategy Change 




02 


12 


-03 


42»»* 




De 1 i berate 




03 


-09 


05 


23* 




Notes 




12 


10 


13 


19* 




Systematic 




06 


09 


24** 


23* 




New Approach 




12 


-13 


-05 


15 



*p < .05 
**p < .01 
***p < .001 
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Table 16 shows means for problem-solving accuracy and time spent 
working on the problems. The problems were relatively difficult in that 
each was correctly solved by only 40 to 50^ of the subjects, 

TABLE 16 



PROBLEM SOLVING ACCURACY AND TIME 





Problem Solving Task 


Session 


Percent of sub- 
jects who correc- 
tly solved problem 


Mean 


Time 


Verbal Problems: 












Ma 1 ice and A 1 ice 


1 


50^ 




5.60 


mi nutes 


Murder I n the Fami 1 y . 


2 


40^ 




6.85 




Numeric Problems: 












Hotel Room Problem 


1 


A5% 




2.52 


ir 


Horse Trading Problem 


2 


A5% 




1 .37 


II 



The correlation of each two-problem summed cluster score with each 
two-problem summed accuracy and time score is presented {n Table 17. Work- 
ing carefully and maintaining good concentration appear to have been help- 
ful in solving the Verbal Problems, while using the same hypothesis over 
and over (Fixated) hindered correct solution. Taking notes and getting 
frustrated took time but did not necessarily affect solving the Verbal 
Problems. On the Numeric Problems, a logical systematic approach was 
helpful in leading to an accurate solution but also added time. Getting 
frustrated, taking notes, changing strategies, and deliberating also 
added time but did not necesiSarily affect solution. Note that the data 
show an interaction between task and processes. Using a systematic 
approach was helpful in solving the Numeric problems, but other strate- 
gies were more helpful in solving the Verbal Problems. 

Conclusions 

There is very little evidence to support the idea that the subjects^ 
problem-solving behavior was a function of cognitive style variables. 
The processing dimensions that were identified are more narrow than style 
dimensions, and did not correlate with each other in a way which would 
support the notion that groups of them represent different aspects of 
the same style. The processing dimensions obtained have a large task 
specific component and do not show the amount of generality that is 
expected from stylistic variables. In addition, practice effects of the 
type consistent with conceptualizations of strategies rather than styles 
were found. Processing was more dependent on task than subject para- 
meters. In short, underlying generalized tendencies in the form of cog- 
nitive styles do not appear to have played a role in determining problem- 
solving behavior. Thus, for several reasons the processes identified in 
subjects' problem-solving behavior are more accurately conceptualized as 
representative of cognitive strategies than co^f^itive styles. 
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These findings suggest that flexibility-rigidity, reflection- 
impulslvity, and global-^ana lytic styles may be restricted in their appli- 
cability to intellectual activities other than deductive-reasoning problem- 
solving. This conclusion necessarily reduces the extent to which cognitive 
style constructs can be considered generalized traits of individuals, and 
causes one to ask If there might be other areas in which styles likewise 
do not play a significant role. In discussing the restrictions on cog- 
nitive style constructs, Kagan and Kogan (1970) point out that the analytic- 
tied principle is largely restricted to performance on spatial tasks and 
not systematically related to verbal processes, Kagan & Kogan point out 
that commonly used measures of ref lection-impu Isi vi ty do not involve ver- 
bal processes, and that this dimension tends to be unrelated to language 
skills. Thus, it is becoming increasingly clear that many currently used 
"cognitive style test^" do not measure general tendencies to process in- 
formation in a way that is consistent across all situations and ail infor- 
mational domains. 

It is more likely that these tests measure aspects of mental pro- 
cessing that are quite specific to the task and siruatlon In which the 
measurement Is taken. The results of this study illustrate the necessity 
of assessing the contribution of situational factors when attempts are 
made to measure hypothesized underlying traits. Also Illustrated is the 
value of taking several measurements in order to assess consistency In 
responding both over time and across different tasks. It is indeed a 
risky business to infer the existence of a generalized trait 1 1 ke a cog- 
nitive style from a single measurement with an instrument which samples 
only one mode of information or dimension of responding. 

An additional finding is that the processes measured in these sub- 
jects were. In several cases, di f ferentl a I ly related to the accuracy and 
time taken on the two types of problem-solving tasks. Thus, the methods 
used to assess problem-solving processes were shown to have some predic- 
tive validity. The finding that, for example, a subject is more likely 
to solve the Numeric Problems if he uses a systematic approach but that 
other strategies are more helpful In solving the Verbal Problems demon- 
strates how unapparent differences between problems can be identified by 
psychometric techniques. The Information gained from this kind of task 
analysis should be useful to researchers and educators who are interested 
In finding ways to improve learning and problem-solving skills. 
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CHAPTER 4 



LEARNING STRATEGiES OF COLLEGE STUDENTS 

College learning is a complex activity that may be approached in 
a variety of ways* When tasks are open ended, the wider is the range of 
potentially effective strategies that may be employed, and that may lead 
to success. C. Frederlksen (1970), termed tasks that admit a number of 
different effective strategies as "functionally Indeterminate." The 
present study is concerned with developing information about the effec- 
tive strategies for college (or any complex and abstract) learning tasks. 
In the past, a common research tool has been the questionnaire, the best 
known of which Is the Survey of Study Habits and Attitudes (SSHA) (Brown 
& Holtzman, 1953, 1955). The SSHA rather than providing a detailed mea- 
sure of learning strategies is a un 1 dimensiona I scale that concentrates 
on motivation and "good" study habits such as wasting no time, consis- 
tently attending lectures, taking good notes, and having a study timetable. 

Biggs (1970) took a different tach, "The situation is complex and 
it is not simplified by using a blanket 'good-bad' dimension to charac- 
terize study habits. The questionnaire approach might be more fruitful 
if a number'of specific and unldimenslonal subscales were produced and 
these could be related to specific persons and task conditions Cp.l6l- 
162]." To this end Biggs constructed a Study Behavior Questionnaire (SBQ) 
that measured operations applied to studying that are a function of per- 
sonality characteristics. The factor structure of the SBQ revealed six 
factors; (a) Study organization, (b) Tol^srance of ambiguity, (c) Cogni- 
tive simplicity, (d) Capacity for Intrinsic motivation, (e) Dogmatism, 
and (f) Independence of study behavior. Rather than specific Informa- 
tion processing dimensions, rather broad motivational and stylistic fac- 
tors were Isolated. 

More recently Goldman (1972a) has shown that self-reports could be 
used to classify college students Into two learning strategy groups; a 
logical group and a mnemonic group. These groups were found to differ 
in success, on several criteria. In two different psychology courses- In 
a series of studies (Goldman & Warren, 1972, (973; Goldman & Hudson, 1973). 
(Soldman and his students have used a Study Techniques Questionnaire (STQ) 
to map strategy and ability differences between males and females, and 
successful and unsuccessful college students, and to discriminate between 
college grade success In different major fields and major field concentra- 
tion In terms of learning strategies. Originally ((5oldman & Warren, 1972) 
eight orthogonal dimensions were obtained, and later seven oblique dtmen-' 
sfons were employed for subscales (Goldman & Hudson, 1973). The eight 
identified dimension:^ are: (a) clerical diligence, (b) academic "savvy", 
(c) mnemonics, (d) planfulness, (e) formal thinking, (f) note taking, 
(g) transformations and applications, and (h) underlining. Our research 
(which was conceptualized independently of Goldman's) can be considered 
as an attempt to extend (Soldman's efforts toward a finer grained analysis 
of strategies than the dimensions employed in hts STQ, which were based 
on a set of items that reflected, in general, a wide variety of study tech- 
niques, goals, and self -evaluations. Two theoretical orientations guided 
our work. First, studying is an active process. Thus, effective study 
strategies are those that Involve some sort of Interaction with or trans- 
formation of the study materials. For example, noting the main points 
In a lecture, as opposed to trying to transcribe every word* The first 
strategy Involves a transformation and reorganization of the Input; the 
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second can be*. cons! de red a passive strategy requiring no actwe processing 
of input. Second, detailed specifications of study processes can be ob- 
tained by the multivariate structural analysis of self-reports obtained 
from separate questionnaires based on spec! fie learning variables such 
as reading, study atmosphere, lecture processing, studying for examina- 
tions^ and memorizing. 

The purpose of our investigation was to determine the feasibility 
of constructing specific si tuatlona I ly based scales and to extract the 
structure of the scales In a pilot Investigation. At the present time, 
on'ly the first objective has been accomplished, and this in a limited way. 
The results of our research, nonetheless, suggest that subjects can de- 
scribe in detail a variety of situational based study procedures and that 
scales based on such reports would be multidimensional and reveal some of 
the fine detail of learning strategies. 

Method 

Subjects . The subjects were undergraduate volunteers from a variety 
of psychology courses at Michigan State University. Approximately 150 
students participated, and about three-fourths of these returned the 
materials. Tht> subjects received one and a half credits toward partici- 
pation In required psychology experiments. 

Procedure . The subjects were told by their Instructors they couid 
take home, fill out, and return to a central location within a week^a 
brief questionnaire surveying how students studied. The survey (Student 
Studying Strategy Survey) material was relatively Instructed and pre- 
sented In a self-paced booklet that took between a half to an hour and a 
half to complete. The survey briefly outlined that the researchers were 
Interested In the students' "studying procedures" and that they were going 
to be asked to answer questions about them. An example of a memoriza- 
tion strategy was presented to Indicate the kind of Information desired. 
The questionnaire had an open-ended format and consisted of three sec- 
tions. Section one asked for a description of "all the things you do 
while studying — use 3-4 sentences to describe each technique*.." Part 
two asked about study procedures In specific situations. Again each re- 
sponse was to be brief, about 3-4 sentences. The situations were Lectures, 
College Reading, Preparation for Examinations, Study Routines, and Study 
Atmosphere. Part three asked the students to look over their previous 
answers and make additions, etc., and to note procedures not asked about 
In the survey. 

Results and Discussion 

The returned surveys were all briefly read, and 70 surveys that had 
fairly complete responses for each section were selected for analysis. 
The analysis procedure consisted of several steps. The first step con- 
sisted of verbatim transcriptions of each Individual's statement of a 
process from about 25 surveys selected at random from the 70. This was 
done separately for each section of the survey and for each specific 
learning situation. The process descriptions transcribed to this point 
were sorted into categories and several statements re-written to Indi- 
cate the main point of the strategy. The remaining surveys were then 
checked, one by one, against the detailed set of statements and categories. 
Any new process statements were recorded^ and statements judged by the 
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experimenters to represent one already recorded were noted by indicating 
frequency. The complete analysis, thus, provided a set of process de- 
scriptions with frequency counts for each statement for 70 surveys. 

The process categories discovered in the first stage of analysis are 
given below for the various parts and learning situations described in 
the survey form. 
Genera I , 

conscious general strategies 

special courses and assignments 

memorization techniques 

ways of building incentive and concentration 

pressure-time relations 

other techniques 
Lecture . 

form of notes 

extent of notes 

what is important to note? 

lectures — genera I features 

pre- lecture routines 
Reading . 

before a complete reading 

ordered procedures 

physical marks 

condensing procedures 

second reading 

units of reading 

difficult reading 

how much reading 

general reading procedures 
Preparation for examinations , 

ordered procedures " 

general features 

relationship of books and lectures 
temporal problems 
other techniques 
when to start 

Routi nes > ^ 
time of day 

difficult course— seek! ng of help 
difficult course — general features 
expenditure of time — general featjures 
structuring of day 
Atmosphere , 

genera! features 

soci a I atmosphere 

phys 1 ca I atmosphere— no i se I eve I 

physical atmosphere — place of study 

physical atmosphere--bod My position and comfort 

phys 1 ca I atmosphere— mi see 1 1 aneous 

other 

Each of these categories was represented by several different pro- 
cess statements. Especially good information was obtained for the lec- 
ture and reading situations. Enough of the statements represented rnf^morl- 
zati on techniques to suggest a separate questionnaire for this aspect of 
Q studying. 
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At this point a preliminary attempt was made to construct some mea- 
suring scales using the data"ana lyzed so far. These were only very crude 
scales serving to summarize our empirical efforts to that time. The 
scales are not reproduced here; their incomplete, unrevised and tenta- 
tive nature preclude their use in any meaningful or. valid way as research 
instruments. Interested readers may correspond with the author for 
additional details. Besides a general scale consistency of 43 i1<3ms, scales 
were constructed for the specHic learning situations: Memorization 
(23 items), Reading (27 items;. Examinations (48 Items), Lecture pro- 
cessing (52 items), and Atmosphere (39 items). Examples of items are: 
try to condense the material into general principles and try to remember 
that, rather than specific points and details,"; "I memorize by breaking 
the material down into a set of small •chunks', which are themselves eaiiier 
to remember,"; "I write down only the general ideas of each lecture,"; "I 
skim over the notes I had taken previously, before a lecture,'*; "I try 
to just get the main ideas during the first reading of a text,"; "I under- 
line especially those points in the book that the professor has stressed in 
class,''; "I ask myself questions as I am going over the material,"; "i 
compress lecture and book notes and study that outline or page." 

These examples taken from the Memorization Scale, the Lecture Pro- 
cessing Scale, the Reading Scale, and the Examinations Scale serve to 
indicate the kind of information we were able to gather and condense into 
scale items. The scales we constructed should not be viewed as useable 
research i nstrurnents at this time; they are I i ml ted in several ways. Per- 
haps the most important limitation is coverage of potentially relevant 
strategies by the present set of items. Many more good items can be 
written than are now on the scales; items that would cover different pro- 
cesses than the present set, A careful analysis of the Item pool has re- 
vealed several gaps and omissions that need to be filled in. One approach 
here has been to analyze various "How to Study Manuals" for process 
suggestions that could be incorporated as scale items. 

The research to date, although limited, has served to Indicate that 
a primarily empirical approach to the development of specific learning 
strategy scales can be successful in uncove, ing a large set of seeming 
different processes. Further development of the materials to extend the 
item pools and refine the scales seems justified by the progress to date. 
Multivariate analysis of the dimensions underlying refined scales should 
reveal a number of basic "learning strategies," 
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CHAPTER 5 



CONCLUSIONS 

Our experiments were concerned with the identification and descrip- 
tion of cognitive processes that mediate performance on complex problem- 
solving and learning tasks. The methods used relied heavily on the ver- 
bal reports of subjects describing what they were doing while working 
on intellectual tasks. The present results strongly suggest that such 
reports, when taken as behavior data to be further analyzed (not as In- 
trospective evidence of mental functioning), provide useful and adequate 
descriptions of some of the component processes of problem solving and 
abstract lea?-ning. In general our results show that learning and problem- 
solving strategies are a function of both task structure and subject 
characteristics. Individual differences between subjects and consisten- 
cies within subjects in the processes employed were found in both the 
anagram and cognitive style experiments* The results, however, indicate 
that task features played the dominant role in determining the processes 
emp loyed . 

These findings support the ''differential process model" of C. Freder- 
iksen (1969). Frederiksen demonstrated a predictable relationship between 
subject chcjracteri sties (e.g., abilities), task structure, and strategy 
choice. In Freder i ksen's research, abilities largely determined strategy 
use in relatively unstructured tasks, whereas the task itself determined 
strategy choice for highly structured tasks. Since our tasks were highly 
structured, the finding of a high degree of task dependency in process use [ 
supports Frederi ksen 's model. 

The finding of a strong dependence between task structure and strat- 
egy "choice" cjIso indicates a type of ATI (Aptitude x Treatment Interac- 
tion) (Cronbach & Snow, 1969). Strategies that are highly effective for 
one task may not be for another. Such a result was obtained in the styles 
experiment. A systematic rational strategy was found effective for nu- 
meric problem solving, while other strategies were found more effective 
for verbal tasks. The specification of cognitive processes would seem 
to be an important direction for ATI research. Tasks can be "mapped" 
for the relevant strategies that are effective for the task, and Indivi- 
dual differences between subjects can be described in terms of differences 
in strategy choice and effectiveness. The work of (5oldman and his colleagues 
(see chapter 4) demonstrates both points. Major fields of study for 
college students could be described in terms of differences in learning 
strategies (Goldman & Hudson, 1973; Goldman & Warren, 1972), and differ- 
ences in study strategies were found to differentiate between successful 
and unsuccessful college students (Goldman & Warren, 1973). As suggested 
by the ATI paradigm, knowledge of the relevant processes for particular 
task domains would allow the design of optimum instructional methods for 
these domains. Likewise, knowledge of individual differences in process 
choices and effectiveness (a process profile) would make it possible to 
assign pupils of differing skills to optimal instructional sequences. 
Process "tests" devised for this purpose could also be used to diagnose 
deficiencies in problem-solving and lecirning strategies in school and 
college learning situations. 

One important purpose of our research was to develop general proce- 
dures for constructing psychometr i ca I ly based measurement scales of cog- 
nitive processes. Two approaches were explored in our research — one em- 
pirical and the other theoretical. The empirical approaches studied were 
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the "protocol analysis" procedures (chapter 2) .'ewe I I and Simon (1972) 
and the more traditional scale construction methods (chapter 4), which in- 
volve generating a large set of potential scale items from the relatively 
unstructured surveying of a large number of subjects. The theoretical 
approach (chapter 3) was explored by constructing scale items based on 
and designed to measure constructs derived from psychological theory. 
The protocol analysis procedures have the advantage of providing fine 
grained descriptions of cognitive functioning that are unusually com- 
plete and formal. The method also provides information about how the 
processes functiopand interact together in the behavior of the processor. 
That is, the structure of processing is also revealed. For our purposes, 
the method's major drawback is time. Protocol analyiiis for even one 
subject takes many weeks to complete and verify. The other empirical 
and theoretical procedures can generate an initial item pool in a very 
short period of time. As a summary of our findings in this aroa, a 
general procedure for the construction of process tests Is sketched below. 

In general, the recommendation is to employ a combination of all 
three approaches. Step one: Pre-analyze the task domain in information 
processing terms. That is, on the basis of relevant I iterature and pro-* 
cess considerations, analyze the task by trying to specify in advance the 
processes and higher level strategies that could be involved In the task. 
Pre-ana lysis of task environments Is an important part of the protocol 
analysis procedures (Newell and Simon, 1972). Protocol analysis could 
be a useful tool here, although on a more restricted and less ambitious 
scale than followed in our research. A few subjects' behavior could be 
analyzed in some detail and process descriptions extracted. The eposide 
analysis procedures recommended by Newell and Simon for f?>ore limited pro- 
tocol analysis would seem to be useful in this step. Step two: Gather 
data from a large number of subjects with an unstructured instrument 
that asks subjects to describe the processes they use in working on the 
tasks (see chapter 4 for details on how this can be accomplished). Step 
three: Generate a large set of Items that describe the processes speci- 
fied and reported on in steos one and two^/ Each item should be written 
to indicate some aspect of a processing mechanism, describing what a per- 
son might d£ in trying to accomplish the task. For example "I condensed 
the lecture material into a set of major points." Step four: Refine 
Items, make structure of each item parallel, replace poor items, remove 
duplications, etc. Step five: Construct scale from the Items. Step six: 
Administer scale(s) to pilot group of subjects and reduce data by factor 
analytic or cluster analytic methods (see chapters one & three) to a set 
of common process dimensions that summarize subjects' responses on the 
scales. Step seven: Define empirical dimensions and refine item struc- 
ture based on pilot analysis. 
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